Collation versioning
Started by Thomas Munroover 7 years ago192 messages
Hello,
While reviewing the ICU versioning work a while back, I mentioned the
idea of using a user-supplied command to get a collversion string for
the libc collation provider. I was reminded about that by recent news
about an upcoming glibc/CLDR resync that is likely to affect
PostgreSQL users (though, I guess, probably only when they do a major
OS upgrade). Here's an experimental patch to try that idea out. For
example, you might set it like this:
libc_collation_version_command = 'md5
/usr/share/locale/@LC_COLLATE@/LC_COLLATE | sed "s/.* = //"'
... or, on a Debian system using the locales package, like this:
libc_collation_version_command = 'dpkg -s locales | grep Version: |
sed "s/Version: //"'
Using the checksum approach, it works like this:
postgres=# alter collation "xx_XX" refresh version;
NOTICE: changing version from b88d621596b7e61337e832f7841066a9 to
7b008442fbaf5dfe7a10fb3d82a634ab
ALTER COLLATION
postgres=# select * from pg_collation where collname = 'xx_XX';
-[ RECORD 1 ]-+---------------------------------
collname | xx_XX
collnamespace | 2200
collowner | 10
collprovider | c
collencoding | 6
collcollate | en_US.UTF-8
collctype | UTF-8
collversion | 7b008442fbaf5dfe7a10fb3d82a634ab
When the collation definition changes you get the desired scary
warning on next attempt to use it in a fresh backend:
postgres=# select * from t order by v;
WARNING: collation "xx_XX" has version mismatch
DETAIL: The collation in the database was created using version
b88d621596b7e61337e832f7841066a9, but the operating system provides
version 7b008442fbaf5dfe7a10fb3d82a634ab.
HINT: Rebuild all objects affected by this collation and run ALTER
COLLATION public."xx_XX" REFRESH VERSION, or build PostgreSQL with the
right library version.
The problem is that it isn't in effect at initdb time so if you add
that later it only affects new locales. You'd need a way to do that
during init to capture the imported system locale versions, and that's
a really ugly string to have to pass into some initdb option. Ugh.
Another approach would be to decide that we're willing to put
non-portable version extracting magic in pg_locale.c. On a long
flight I hacked my libc to store a version string (based on CLDR
version or whatever) in its binary locale definitions and provide a
proper interface to ask for it, modelled on querylocale(3):
const char *querylocaleversion(int mask, locale_t locale);
Then the patch for pg_locale.c is trivial, see attached. While I
could conceivably try to convince my local friendly OS to take such a
patch, the real question is how to deal with glibc. Does anyone know
of a way to extract a version string from glibc using existing
interfaces? I heard there was an undocumented way but I haven't been
able to find it -- probably because I was, erm, looking in the
documentation.
Or maybe this isn't worth bothering with, and we should just build out
the ICU support and then make it the default and be done with it.
In passing, here's a patch to add tab completion for ALTER COLLATION
... REFRESH VERSION.
--
Thomas Munro
http://www.enterprisedb.com
Attachments:
0001-Auto-complete-ALTER-COLLATION-.-REFRESH-VERSION.patchapplication/octet-stream; name=0001-Auto-complete-ALTER-COLLATION-.-REFRESH-VERSION.patchDownload
From 4c975c37d86eb6ca4fc3676e5c6359787ecf97fc Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Tue, 4 Sep 2018 04:57:04 +1200
Subject: [PATCH] Auto-complete ALTER COLLATION ... REFRESH VERSION.
---
src/bin/psql/tab-complete.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index bb696f8ee90..1e79996a855 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1771,7 +1771,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER COLLATION <name> */
else if (Matches3("ALTER", "COLLATION", MatchAny))
- COMPLETE_WITH_LIST3("OWNER TO", "RENAME TO", "SET SCHEMA");
+ COMPLETE_WITH_LIST4("OWNER TO", "RENAME TO", "SET SCHEMA",
+ "REFRESH VERSION");
/* ALTER CONVERSION <name> */
else if (Matches3("ALTER", "CONVERSION", MatchAny))
--
2.17.0
0001-Add-libc_collation_version_command-GUC.patchapplication/octet-stream; name=0001-Add-libc_collation_version_command-GUC.patchDownload
From ce89d0570702c5581100ba16742cc0b6914db61b Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Tue, 4 Sep 2018 04:54:56 +1200
Subject: [PATCH 1/2] Add libc_collation_version_command GUC.
---
src/backend/utils/adt/pg_locale.c | 50 +++++++++++++++++++++++++++++++
src/backend/utils/misc/guc.c | 13 ++++++++
src/include/utils/guc.h | 1 +
3 files changed, 64 insertions(+)
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index a3dc3be5a87..bb65c075676 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -56,7 +56,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "lib/stringinfo.h"
#include "mb/pg_wchar.h"
+#include "storage/fd.h"
#include "utils/builtins.h"
#include "utils/hsearch.h"
#include "utils/lsyscache.h"
@@ -1475,6 +1477,54 @@ get_collation_actual_version(char collprovider, const char *collcollate)
}
else
#endif
+ if (libc_collation_version_command[0] != '\0')
+ {
+ char buffer[1024];
+ StringInfoData output;
+ FILE *file;
+ size_t len;
+ const char *p;
+
+#define LC_COLLATE_TOKEN "@LC_COLLATE@"
+#define LC_COLLATE_TOKEN_LEN (sizeof(LC_COLLATE_TOKEN) - 1)
+
+ /* Build the complete command. */
+ initStringInfo(&output);
+ p = libc_collation_version_command;
+ while (*p)
+ {
+ if (strncmp(p, LC_COLLATE_TOKEN, LC_COLLATE_TOKEN_LEN) == 0)
+ {
+ appendStringInfoString(&output, collcollate);
+ p += LC_COLLATE_TOKEN_LEN;
+ }
+ else
+ appendStringInfoChar(&output, *p++);
+ }
+
+ /* Execute it and read one line. */
+ file = OpenPipeStream(output.data, "r");
+ if (!file)
+ ereport(ERROR,
+ (errmsg("could not run command \"%s\": %m",
+ output.data)));
+ if (!fgets(buffer, sizeof(buffer), file))
+ {
+ ClosePipeStream(file);
+ ereport(ERROR,
+ (errmsg("could not read output from command \"%s\": %m",
+ output.data)));
+ }
+ ClosePipeStream(file);
+ pfree(output.data);
+
+ /* Trim off newline. */
+ len = strlen(buffer);
+ if (len > 0 && buffer[len - 1] == '\n')
+ buffer[len - 1] = '\0';
+ collversion = pstrdup(buffer);
+ }
+ else
collversion = NULL;
return collversion;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 0625eff2191..ed19a71f319 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -474,6 +474,7 @@ char *ConfigFileName;
char *HbaFileName;
char *IdentFileName;
char *external_pid_file;
+char *libc_collation_version_command;
char *pgstat_temp_directory;
@@ -3883,6 +3884,18 @@ static struct config_string ConfigureNamesString[] =
check_cluster_name, NULL, NULL
},
+ {
+ {"libc_collation_version_command", PGC_POSTMASTER, PROCESS_TITLE,
+ gettext_noop("Command to obtain version strings for libc collations."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &libc_collation_version_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+
{
{"wal_consistency_checking", PGC_SUSET, DEVELOPER_OPTIONS,
gettext_noop("Sets the WAL resource managers for which WAL consistency checks are done."),
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index f462eabe594..7efbe033d22 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -262,6 +262,7 @@ extern PGDLLIMPORT char *ConfigFileName;
extern char *HbaFileName;
extern char *IdentFileName;
extern char *external_pid_file;
+extern char *libc_collation_version_command;
extern PGDLLIMPORT char *application_name;
--
2.17.0
0001-Add-libc_collation_version_command-GUC.patchapplication/octet-stream; name=0001-Add-libc_collation_version_command-GUC.patchDownload
From ce89d0570702c5581100ba16742cc0b6914db61b Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Tue, 4 Sep 2018 04:54:56 +1200
Subject: [PATCH 1/2] Add libc_collation_version_command GUC.
---
src/backend/utils/adt/pg_locale.c | 50 +++++++++++++++++++++++++++++++
src/backend/utils/misc/guc.c | 13 ++++++++
src/include/utils/guc.h | 1 +
3 files changed, 64 insertions(+)
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index a3dc3be5a87..bb65c075676 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -56,7 +56,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "lib/stringinfo.h"
#include "mb/pg_wchar.h"
+#include "storage/fd.h"
#include "utils/builtins.h"
#include "utils/hsearch.h"
#include "utils/lsyscache.h"
@@ -1475,6 +1477,54 @@ get_collation_actual_version(char collprovider, const char *collcollate)
}
else
#endif
+ if (libc_collation_version_command[0] != '\0')
+ {
+ char buffer[1024];
+ StringInfoData output;
+ FILE *file;
+ size_t len;
+ const char *p;
+
+#define LC_COLLATE_TOKEN "@LC_COLLATE@"
+#define LC_COLLATE_TOKEN_LEN (sizeof(LC_COLLATE_TOKEN) - 1)
+
+ /* Build the complete command. */
+ initStringInfo(&output);
+ p = libc_collation_version_command;
+ while (*p)
+ {
+ if (strncmp(p, LC_COLLATE_TOKEN, LC_COLLATE_TOKEN_LEN) == 0)
+ {
+ appendStringInfoString(&output, collcollate);
+ p += LC_COLLATE_TOKEN_LEN;
+ }
+ else
+ appendStringInfoChar(&output, *p++);
+ }
+
+ /* Execute it and read one line. */
+ file = OpenPipeStream(output.data, "r");
+ if (!file)
+ ereport(ERROR,
+ (errmsg("could not run command \"%s\": %m",
+ output.data)));
+ if (!fgets(buffer, sizeof(buffer), file))
+ {
+ ClosePipeStream(file);
+ ereport(ERROR,
+ (errmsg("could not read output from command \"%s\": %m",
+ output.data)));
+ }
+ ClosePipeStream(file);
+ pfree(output.data);
+
+ /* Trim off newline. */
+ len = strlen(buffer);
+ if (len > 0 && buffer[len - 1] == '\n')
+ buffer[len - 1] = '\0';
+ collversion = pstrdup(buffer);
+ }
+ else
collversion = NULL;
return collversion;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 0625eff2191..ed19a71f319 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -474,6 +474,7 @@ char *ConfigFileName;
char *HbaFileName;
char *IdentFileName;
char *external_pid_file;
+char *libc_collation_version_command;
char *pgstat_temp_directory;
@@ -3883,6 +3884,18 @@ static struct config_string ConfigureNamesString[] =
check_cluster_name, NULL, NULL
},
+ {
+ {"libc_collation_version_command", PGC_POSTMASTER, PROCESS_TITLE,
+ gettext_noop("Command to obtain version strings for libc collations."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &libc_collation_version_command,
+ "",
+ NULL, NULL, NULL
+ },
+
+
{
{"wal_consistency_checking", PGC_SUSET, DEVELOPER_OPTIONS,
gettext_noop("Sets the WAL resource managers for which WAL consistency checks are done."),
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index f462eabe594..7efbe033d22 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -262,6 +262,7 @@ extern PGDLLIMPORT char *ConfigFileName;
extern char *HbaFileName;
extern char *IdentFileName;
extern char *external_pid_file;
+extern char *libc_collation_version_command;
extern PGDLLIMPORT char *application_name;
--
2.17.0
Re: Collation versioning
On Tue, Sep 4, 2018 at 10:02 AM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
const char *querylocaleversion(int mask, locale_t locale);
Then the patch for pg_locale.c is trivial, see attached.
Oops, here's that one, FWIW.
--
Thomas Munro
http://www.enterprisedb.com
Attachments:
0001-Use-querylocaleversion-3-if-available.patchapplication/octet-stream; name=0001-Use-querylocaleversion-3-if-available.patchDownload
From 94705158a009292bc0ae2f87f65c58bfc0a4901e Mon Sep 17 00:00:00 2001
From: Thomas Munro <munro@ip9.org>
Date: Sat, 11 Aug 2018 08:55:10 +1200
Subject: [PATCH] Use querylocaleversion(3) if available.
For collations provided by libc, get a version string using
querylocaleversion(3) on systems that have it.
*** THIS IS A FICTIONAL LIBC FEATURE ***
Author: Thomas Munro
---
configure | 2 +-
configure.in | 2 +-
src/backend/utils/adt/pg_locale.c | 15 +++++++++++++++
src/include/pg_config.h.in | 3 +++
4 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/configure b/configure
index 26652133d5..49fa863a47 100755
--- a/configure
+++ b/configure
@@ -14916,7 +14916,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
+for ac_func in cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np querylocaleversion readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index 397f6bc765..224385f573 100644
--- a/configure.in
+++ b/configure.in
@@ -1540,7 +1540,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime dlopen fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np querylocaleversion readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index a3dc3be5a8..d89935680f 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1475,7 +1475,22 @@ get_collation_actual_version(char collprovider, const char *collcollate)
}
else
#endif
+#if HAVE_QUERYLOCALEVERSION
+ {
+ locale_t loc;
+
+ loc = newlocale(LC_COLLATE, collcollate, NULL);
+ if (loc)
+ {
+ collversion = pstrdup(querylocaleversion(LC_COLLATE, loc));
+ freelocale(loc);
+ }
+ else
+ collversion = NULL;
+ }
+#else
collversion = NULL;
+#endif
return collversion;
}
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index b7e469670f..d4b077424b 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -449,6 +449,9 @@
/* Have PTHREAD_PRIO_INHERIT. */
#undef HAVE_PTHREAD_PRIO_INHERIT
+/* Define to 1 if you have the `querylocaleversion' function. */
+#undef HAVE_QUERYLOCALEVERSION
+
/* Define to 1 if you have the `random' function. */
#undef HAVE_RANDOM
--
2.18.0
Re: Collation versioning
Re: Thomas Munro 2018-09-04 <CAEepm=0uEQCpfq_+LYFBdArCe4Ot98t1aR4eYiYTe=yavQygiQ@mail.gmail.com>
I was reminded about that by recent news
about an upcoming glibc/CLDR resync that is likely to affect
PostgreSQL users (though, I guess, probably only when they do a major
OS upgrade).
Or replicating/restoring a database to a newer host.
... or, on a Debian system using the locales package, like this:
libc_collation_version_command = 'dpkg -s locales | grep Version: |
sed "s/Version: //"'
Ugh. This sounds horribly easy to get wrong on the user side. I could
of course put that preconfigured into the Debian packages, but that
would leave everyone not using any of the standard distro packagings
in the rain.
Does anyone know
of a way to extract a version string from glibc using existing
interfaces? I heard there was an undocumented way but I haven't been
able to find it -- probably because I was, erm, looking in the
documentation.
That sounds more robust. Googling around:
https://www.linuxquestions.org/questions/linux-software-2/how-to-check-glibc-version-263103/
#include <stdio.h>
#include <gnu/libc-version.h>
int main (void) { puts (gnu_get_libc_version ()); return 0; }
$ ./a.out
2.27
Hopefully that version info is fine-grained enough.
Christoph
Re: Collation versioning
On Wed, Sep 5, 2018 at 3:35 AM Christoph Berg <myon@debian.org> wrote:
Re: Thomas Munro 2018-09-04 <CAEepm=0uEQCpfq_+LYFBdArCe4Ot98t1aR4eYiYTe=yavQygiQ@mail.gmail.com>
I was reminded about that by recent news
about an upcoming glibc/CLDR resync that is likely to affect
PostgreSQL users (though, I guess, probably only when they do a major
OS upgrade).Or replicating/restoring a database to a newer host.
Yeah.
Does anyone know
of a way to extract a version string from glibc using existing
interfaces? I heard there was an undocumented way but I haven't been
able to find it -- probably because I was, erm, looking in the
documentation.That sounds more robust. Googling around:
https://www.linuxquestions.org/questions/linux-software-2/how-to-check-glibc-version-263103/
#include <stdio.h>
#include <gnu/libc-version.h>
int main (void) { puts (gnu_get_libc_version ()); return 0; }$ ./a.out
2.27Hopefully that version info is fine-grained enough.
Hmm. I was looking for locale data version, not libc.so itself. I
realise they come ultimately from the same source package, but are the
locale definitions and libc6 guaranteed to be updated at the same
time? I see that the locales package depends on libc-bin >> 2.27 (no
upper bound), which in turn depends on libc6 >> 2.27, << 2.28. So
perhaps you can have a system with locales 2.27 and libc6 2.28? I
also wonder if there are some configurations where they could get out
of sync because of manual use of locale-gen or something like that.
Clearly most systems would have them in sync through apt-get upgrade
though, so maybe gnu_get_libc_version() would work well in practice?
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
On Wed, Sep 5, 2018 at 8:20 AM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
On Wed, Sep 5, 2018 at 3:35 AM Christoph Berg <myon@debian.org> wrote:
int main (void) { puts (gnu_get_libc_version ()); return 0; }
$ ./a.out
2.27Hmm. I was looking for locale data version, not libc.so itself. I
realise they come ultimately from the same source package, but are the
locale definitions and libc6 guaranteed to be updated at the same
time?
And even if they are, what if your cluster is still running and still
has the older libc.so.6 mapped in? Newly forked backends will see new
locale data but gnu_get_libc_version() will return the old string.
(Pointed out off-list by Andres.) Eventually you restart your cluster
and start seeing the error.
So, it's not ideal but perhaps worth considering on the grounds that
it's better than nothing?
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
Re: Thomas Munro 2018-09-05 <CAEepm=3a5BC7CwsXZo3V4fw6YuAMT2nJ1krwtqOatb=vDqRWEA@mail.gmail.com>
Hopefully that version info is fine-grained enough.
Hmm. I was looking for locale data version, not libc.so itself. I
realise they come ultimately from the same source package, but are the
locale definitions and libc6 guaranteed to be updated at the same
time? I see that the locales package depends on libc-bin >> 2.27 (no
upper bound), which in turn depends on libc6 >> 2.27, << 2.28. So
perhaps you can have a system with locales 2.27 and libc6 2.28?
No because libc6.deb "breaks" locales << 2.27:
Package: libc6
Source: glibc
Version: 2.27-5
Breaks: [...] locales (<< 2.27), locales-all (<< 2.27),
(I can't tell off-hand why this isn't just a stricter dependency in
locales.deb, but it's probably because this variant works better for
upgrades.)
I also wonder if there are some configurations where they could get out
of sync because of manual use of locale-gen or something like that.
Clearly most systems would have them in sync through apt-get upgrade
though, so maybe gnu_get_libc_version() would work well in practice?
I'd hope so. I'm more worried about breakage because of fixes applied
within one glibc version (2.27-5 vs 2.27-6), but I guess Debian's
glibc maintainers are clueful enough not to do that.
Re: Thomas Munro 2018-09-05 <CAEepm=0hoACQLFn8ro7jCO9-wTth2mXXS3K=s09gxKqN2jy8pA@mail.gmail.com>
And even if they are, what if your cluster is still running and still
has the older libc.so.6 mapped in? Newly forked backends will see new
locale data but gnu_get_libc_version() will return the old string.
(Pointed out off-list by Andres.) Eventually you restart your cluster
and start seeing the error.
That problem isn't protected against by PG itself. I've seen clusters
that were upgraded on disk but not restarted yet where every plpgsql
invocation was throwing symbol errors. So I guess we don't have to try
harder for libc.
So, it's not ideal but perhaps worth considering on the grounds that
it's better than nothing?
Ack.
Christoph
Import Notes
Reply to msg id not found: CAEepm0hoACQLFn8ro7jCO9-wTth2mXXS3Ks09gxKqN2jy8pA@mail.gmail.comCAEepm3a5BC7CwsXZo3V4fw6YuAMT2nJ1krwtqOatbvDqRWEA@mail.gmail.com | Resolved by subject fallback
Re: Collation versioning
On Wed, Sep 5, 2018 at 12:10 PM Christoph Berg <myon@debian.org> wrote:
So, it's not ideal but perhaps worth considering on the grounds that
it's better than nothing?Ack.
Ok, here's a little patch like that.
postgres=# select collname, collcollate, collversion from pg_collation
where collname = 'en_NZ';
collname | collcollate | collversion
----------+-------------+-------------
en_NZ | en_NZ.utf8 | 2.24
(1 row)
--
Thomas Munro
http://www.enterprisedb.com
Attachments:
0001-Set-collversion-for-collations-that-come-from-glibc.patchapplication/octet-stream; name=0001-Set-collversion-for-collations-that-come-from-glibc.patchDownload
From a05d579957338ca30c01f780c19bed743c94dcb2 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Thu, 6 Sep 2018 09:05:57 +1200
Subject: [PATCH] Set collversion for collations that come from glibc.
On systems using glibc (most GNU/Linux distributions), we can defend
against index corruption by using the glibc version string to detect
potential changes in collation definitions.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
configure | 2 +-
configure.in | 2 +-
src/backend/utils/adt/pg_locale.c | 13 +++++++++++++
src/include/pg_config.h.in | 3 +++
4 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/configure b/configure
index dd94c5bbab..faa0a067ef 100755
--- a/configure
+++ b/configure
@@ -12671,7 +12671,7 @@ $as_echo "#define HAVE_STDBOOL_H 1" >>confdefs.h
fi
-for ac_header in atomic.h crypt.h fp_class.h getopt.h ieeefp.h ifaddrs.h langinfo.h mbarrier.h poll.h sys/epoll.h sys/ipc.h sys/prctl.h sys/procctl.h sys/pstat.h sys/resource.h sys/select.h sys/sem.h sys/shm.h sys/sockio.h sys/tas.h sys/un.h termios.h ucred.h utime.h wchar.h wctype.h
+for ac_header in atomic.h crypt.h fp_class.h getopt.h gnu/libc-version.h ieeefp.h ifaddrs.h langinfo.h mbarrier.h poll.h sys/epoll.h sys/ipc.h sys/prctl.h sys/procctl.h sys/pstat.h sys/resource.h sys/select.h sys/sem.h sys/shm.h sys/sockio.h sys/tas.h sys/un.h termios.h ucred.h utime.h wchar.h wctype.h
do :
as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default"
diff --git a/configure.in b/configure.in
index 3280afa0da..03cbb78bfa 100644
--- a/configure.in
+++ b/configure.in
@@ -1265,7 +1265,7 @@ AC_SUBST(UUID_LIBS)
AC_HEADER_STDBOOL
-AC_CHECK_HEADERS([atomic.h crypt.h fp_class.h getopt.h ieeefp.h ifaddrs.h langinfo.h mbarrier.h poll.h sys/epoll.h sys/ipc.h sys/prctl.h sys/procctl.h sys/pstat.h sys/resource.h sys/select.h sys/sem.h sys/shm.h sys/sockio.h sys/tas.h sys/un.h termios.h ucred.h utime.h wchar.h wctype.h])
+AC_CHECK_HEADERS([atomic.h crypt.h fp_class.h getopt.h gnu/libc-version.h ieeefp.h ifaddrs.h langinfo.h mbarrier.h poll.h sys/epoll.h sys/ipc.h sys/prctl.h sys/procctl.h sys/pstat.h sys/resource.h sys/select.h sys/sem.h sys/shm.h sys/sockio.h sys/tas.h sys/un.h termios.h ucred.h utime.h wchar.h wctype.h])
# On BSD, test for net/if.h will fail unless sys/socket.h
# is included first.
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index a3dc3be5a8..332655877e 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -68,6 +68,10 @@
#include <unicode/ucnv.h>
#endif
+#ifdef HAVE_GNU_LIBC_VERSION_H
+#include <gnu/libc-version.h>
+#endif
+
#ifdef WIN32
/*
* This Windows file defines StrNCpy. We don't need it here, so we undefine
@@ -1475,6 +1479,15 @@ get_collation_actual_version(char collprovider, const char *collcollate)
}
else
#endif
+ if (collprovider == COLLPROVIDER_LIBC)
+ {
+#ifdef HAVE_GNU_LIBC_VERSION_H
+ collversion = gnu_get_libc_version();
+#else
+ collversion = NULL;
+#endif
+ }
+ else
collversion = NULL;
return collversion;
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 347d5b56dc..55025090b6 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -282,6 +282,9 @@
/* Define to 1 if you have the `gettimeofday' function. */
#undef HAVE_GETTIMEOFDAY
+/* Define to 1 if you have the <gnu/libc-version.h> header file. */
+#undef HAVE_GNU_LIBC_VERSION_H
+
/* Define to 1 if you have the <gssapi/gssapi.h> header file. */
#undef HAVE_GSSAPI_GSSAPI_H
--
2.17.0
Re: Collation versioning
On 05/09/2018 23:18, Thomas Munro wrote:
On Wed, Sep 5, 2018 at 12:10 PM Christoph Berg <myon@debian.org> wrote:
So, it's not ideal but perhaps worth considering on the grounds that
it's better than nothing?Ack.
Ok, here's a little patch like that.
postgres=# select collname, collcollate, collversion from pg_collation
where collname = 'en_NZ';
collname | collcollate | collversion
----------+-------------+-------------
en_NZ | en_NZ.utf8 | 2.24
(1 row)
But wouldn't that also have the effect that glibc updates that don't
change anything about the locales would trigger the version
incompatibility warning?
Also, note that this mechanism only applies to collation objects, not to
database-global locales. So most users wouldn't be helped by this approach.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On Thu, Sep 6, 2018 at 12:01 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 05/09/2018 23:18, Thomas Munro wrote:
On Wed, Sep 5, 2018 at 12:10 PM Christoph Berg <myon@debian.org> wrote:
So, it's not ideal but perhaps worth considering on the grounds that
it's better than nothing?Ack.
Ok, here's a little patch like that.
postgres=# select collname, collcollate, collversion from pg_collation
where collname = 'en_NZ';
collname | collcollate | collversion
----------+-------------+-------------
en_NZ | en_NZ.utf8 | 2.24
(1 row)But wouldn't that also have the effect that glibc updates that don't
change anything about the locales would trigger the version
incompatibility warning?
Right. And likewise, a glibc update that does change some locales but
not the locales that you are actually using will trigger false alarm
warnings. The same goes for the ICU provider, which appears to return
the same collversion for every collation, even though presumably some
don't change from one ICU version to the next.
I wonder if someone here knows how many "locales" packages have been
released over the lifetime of (say) the current Debian stable distro,
whether any LC_COLLATE files changed over those releases, and whether
libc6 had the same MAJOR.MINOR for the whole lifetime. That is, even
though they might have been through 2.19-17+blah, 2.19-18+blah, ...
did they all report "2.19" and were the collations actually stable?
If that's the case, I think it'd be quite good: we'd only raise the
alarm after a big dist-upgrade Debian 8->9, or when doing streaming
replication from a Debian 8 box to a Debian 9 box.
Also, note that this mechanism only applies to collation objects, not to
database-global locales. So most users wouldn't be helped by this approach.
Yeah, right, that would have to work for this to be useful. I will
look into that.
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
Fwiw, I was doing some tests with LC_COLLATE last year:
https://github.com/ChristophBerg/lc_collate_testsuite
Iirc the outcome was that everything except de_DE.UTF-8 was stable.
Christoph
Re: Collation versioning
On Thu, Sep 6, 2018 at 5:36 PM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
On Thu, Sep 6, 2018 at 12:01 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:Also, note that this mechanism only applies to collation objects, not to
database-global locales. So most users wouldn't be helped by this approach.Yeah, right, that would have to work for this to be useful. I will
look into that.
We could perform a check up front in (say) CheckMyDatabase(), or maybe
defer until the first string comparison. The tricky question is where
to store it.
1. We could add datcollversion to pg_database.
2. We could remove datcollate and datctype and instead store a
collation OID. I'm not sure what problems would come up, but for
starters it seems a bit weird to have a shared catalog pointing to
rows in a non-shared catalog.
The same question comes up if we want to support ICU as a database
level default. Add datcollprovider, or point to a pg_collation row?
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
On 07/09/2018 23:34, Thomas Munro wrote:
On Thu, Sep 6, 2018 at 5:36 PM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:On Thu, Sep 6, 2018 at 12:01 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:Also, note that this mechanism only applies to collation objects, not to
database-global locales. So most users wouldn't be helped by this approach.Yeah, right, that would have to work for this to be useful. I will
look into that.We could perform a check up front in (say) CheckMyDatabase(), or maybe
defer until the first string comparison. The tricky question is where
to store it.1. We could add datcollversion to pg_database.
2. We could remove datcollate and datctype and instead store a
collation OID. I'm not sure what problems would come up, but for
starters it seems a bit weird to have a shared catalog pointing to
rows in a non-shared catalog.The same question comes up if we want to support ICU as a database
level default. Add datcollprovider, or point to a pg_collation row?
This was previously discussed here:
/messages/by-id/f689322a-4fc5-10cc-4a60-81f1ff0166c9@2ndquadrant.com
-- without a conclusion.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
Re: Thomas Munro 2018-09-07 <CAEepm=1xGTsLDx63UEdcJ8MdG63CNJ-tsDWHbH9djtvxRH5ZWw@mail.gmail.com>
2. We could remove datcollate and datctype and instead store a
collation OID. I'm not sure what problems would come up, but for
starters it seems a bit weird to have a shared catalog pointing to
rows in a non-shared catalog.
Naive idea: make that catalog shared? Collations are system-wide after
all.
Christoph
Re: Collation versioning
On 12/09/2018 10:15, Christoph Berg wrote:
Re: Thomas Munro 2018-09-07 <CAEepm=1xGTsLDx63UEdcJ8MdG63CNJ-tsDWHbH9djtvxRH5ZWw@mail.gmail.com>
2. We could remove datcollate and datctype and instead store a
collation OID. I'm not sure what problems would come up, but for
starters it seems a bit weird to have a shared catalog pointing to
rows in a non-shared catalog.Naive idea: make that catalog shared? Collations are system-wide after
all.
By the same argument, extensions should be shared, but they are not.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
Re: Peter Eisentraut 2018-09-12 <0447ec7b-cdb6-7252-7943-88a4664e7bb7@2ndquadrant.com>
Naive idea: make that catalog shared? Collations are system-wide after
all.By the same argument, extensions should be shared, but they are not.
But extensions put a lot of visible stuff into a database, whereas a
collation is just a line in some table that doesn't get into the way.
Christoph
Re: Collation versioning
On 12/09/2018 13:25, Christoph Berg wrote:
Re: Peter Eisentraut 2018-09-12 <0447ec7b-cdb6-7252-7943-88a4664e7bb7@2ndquadrant.com>
Naive idea: make that catalog shared? Collations are system-wide after
all.By the same argument, extensions should be shared, but they are not.
But extensions put a lot of visible stuff into a database, whereas a
collation is just a line in some table that doesn't get into the way.
How about C functions? They are just a system catalog representation of
something that exists on the OS.
Anyway, we also want to support application-specific collation
definitions, so that users can CREATE COLLATION
"my_specific_requirements" and use that that in their application, so
global collations wouldn't be appropriate for that.
Moreover, the fix for a collation version mismatch is, in the simplest
case, to go around and REINDEX everything. Making the collation or
collation version global doesn't fix that. It would actually make it
harder because you couldn't run ALTER COLLATION REFRESH VERSION until
after you have rebuilt all affected objects *in all databases*.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
Re: Peter Eisentraut 2018-09-13 <4f60612c-a7b5-092d-1532-21ff7a106bd5@2ndquadrant.com>
Moreover, the fix for a collation version mismatch is, in the simplest
case, to go around and REINDEX everything. Making the collation or
collation version global doesn't fix that. It would actually make it
harder because you couldn't run ALTER COLLATION REFRESH VERSION until
after you have rebuilt all affected objects *in all databases*.
Btw, I think a "reindexdb --all --collation" (and the SQL per-database
equivalent) that only rebuilds indexes that are affected by collations
would be immensely useful to have.
Christoph
Re: Collation versioning
Greetings,
* Christoph Berg (myon@debian.org) wrote:
Re: Peter Eisentraut 2018-09-13 <4f60612c-a7b5-092d-1532-21ff7a106bd5@2ndquadrant.com>
Moreover, the fix for a collation version mismatch is, in the simplest
case, to go around and REINDEX everything. Making the collation or
collation version global doesn't fix that. It would actually make it
harder because you couldn't run ALTER COLLATION REFRESH VERSION until
after you have rebuilt all affected objects *in all databases*.Btw, I think a "reindexdb --all --collation" (and the SQL per-database
equivalent) that only rebuilds indexes that are affected by collations
would be immensely useful to have.
As I was discussing w/ Peter G during PostgresOpen, we'd have to wait
until that reindexdb is complete before actually using anything in the
system and that's pretty painful. While it sounds like it'd be a good
bit of work, it seems like we really need to have a way to support
multiple collation versions concurrently and to do that we'll need to
have the library underneath actually providing that to us. Once we have
that, we can build new indexes concurrently and swap to them (or,
ideally, just issue REINDEX CONCURRENTLY once we support that..).
Until then, it seems like we really need to have a way to realize that a
given upgrade is going to require a big reindex, before actually doing
the reindex and suddenly discovering that we can't use a bunch of
indexes because they're out of date and extending the downtime for the
upgrade to be however long it takes to rebuild those indexes...
Thanks!
Stephen
Re: Collation versioning
On Thu, Sep 13, 2018 at 7:03 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 12/09/2018 13:25, Christoph Berg wrote:
Re: Peter Eisentraut 2018-09-12 <0447ec7b-cdb6-7252-7943-88a4664e7bb7@2ndquadrant.com>
Naive idea: make that catalog shared? Collations are system-wide after
all.By the same argument, extensions should be shared, but they are not.
But extensions put a lot of visible stuff into a database, whereas a
collation is just a line in some table that doesn't get into the way.How about C functions? They are just a system catalog representation of
something that exists on the OS.Anyway, we also want to support application-specific collation
definitions, so that users can CREATE COLLATION
"my_specific_requirements" and use that that in their application, so
global collations wouldn't be appropriate for that.Moreover, the fix for a collation version mismatch is, in the simplest
case, to go around and REINDEX everything. Making the collation or
collation version global doesn't fix that. It would actually make it
harder because you couldn't run ALTER COLLATION REFRESH VERSION until
after you have rebuilt all affected objects *in all databases*.
Here's one idea I came up with. It involves a new kind of magic. The
goals are:
1. Support versioning for the libc provider, including for the
default collation.
2. Support ICU for the default collation.
3. Fix the tracking of when reindexes need to be rebuilt, so that you
can't get it wrong (as you're alluding to above).
Changes:
1. Drop the datcollate and datctype columns from pg_database.
2. In CheckMyDatabase() or elsewhere in backend initialisation, get
that information instead by loading the pg_collation row with OID =
DEFAULT_COLLATION_OID.
3. Don't put COLLPROVIDER_DEFAULT into the default collation
collprovider column, instead give it a concrete provider value, ie
COLLPROVIDER_LIBC.
4. After creating a new database, update that row as appropriate in
the new database (!). Or find some other way to write a new table out
and switch it around, or something like that. That is, if you say
CREATE DATABASE foo LC_COLLATE = 'xx_XX', COLLATION_PROVIDER = libc
then those values somehow get written into the default pg_collation
row in the *new* database (so at that point it's not a simple copy of
the template database).
5. Drop the collversion column from pg_collation. Get rid of the
REFRESH VERSION command. Instead, add a new column indcollversion to
pg_index. It needs to be an array of text (not sure if that is a
problem in a catalog), with elements that correspond to the elements
of indcollation.
6. Do the check and log warnings when we first open each index.
7. Update indcollversion at index creation and whenever we REINDEX.
I haven't actually tried any of this so I'm not sure if I'm missing
something other than the inherent difficulty of updating a row in a
table in a database you're not connected to...
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
On Sun, Sep 16, 2018 at 1:20 AM Thomas Munro <thomas.munro@enterprisedb.com>
wrote:
3. Fix the tracking of when reindexes need to be rebuilt, so that you
can't get it wrong (as you're alluding to above).
I've mentioned this in the past, but didn't seem to get any traction, so
I'll try it again ;-)
The focus on indexes when a collation changes is, in my opinion, the least
of the problems. You definitely have to worry about indexes, but they can
be easily rebuilt. What about other places where collation is hardened into
the system, such as constraints?
For example, in ICU 4.6 the handling of accents changed for French.
Previously accents were considered right-to-left but ICU 4.6 reversed this.
So consider a constraint like CHECK COL < 'coté' (last letter is U+00E9,
small letter e with acute). Prior to ICU 4.6 the value 'côte' (second
letter is U+00F4, small letter o with circumflex) would have passed this
constraint. With 4.6 or later it would be rejected because of the accent
ordering change. As soon as the collation changes, this table becomes
inconsistent and a reindex isn't going to help it. This becomes a data
cleansing problem at this point (which sucks for the user because their
data was clean immediately prior to the "upgrade").
There have similarly been cases where ICU changes have caused equal
characters to become unequal and vice versa. (Unfortunately all my ICU
notes with examples are at my previous employer.) Consider the effect on RI
constraints. The primary key can be fixed with a reindex (although dealing
with two existing values becoming equal is a pain). But then the user also
has to deal with the foreign keys since they may now have foreign keys
which have no match in the primary key.
And constraints problems are even easier than triggers. Consider a database
with complex BI rules that are implemented through triggers that fire when
values are/are not equal. If the equality of strings change, there could be
bad data throughout the tables. (At least with constraints the inter-column
dependencies are explicit in the catalogs. With triggers anything goes.)
All this collation stuff is great, and I know users want it, but it feels
like were pushing them out of an airplane with a ripped parachute every
time the collation libraries change. Maybe they'll land safely or maybe
things will get very messy.
- Doug
Salesforce
Re: Collation versioning
"Douglas" == Douglas Doole <dougdoole@gmail.com> writes:
Douglas> And constraints problems are even easier than triggers.
Douglas> Consider a database with complex BI rules that are implemented
Douglas> through triggers that fire when values are/are not equal. If
Douglas> the equality of strings change, there could be bad data
Douglas> throughout the tables.
Perhaps fortunately, collation changes cannot (in PG) affect the
equality or non-equality of strings (at least of text/varchar/char
types, citext is a different matter). For the builtin string types, PG
follows the rule that if the collation calls the values equal, they are
ordered secondarily in codepoint order; only byte-identical values can
actually be equal (we need this in order for hashing to work in the
absence of a strxfrm implementation that we can trust).
(This is the same rule used for string comparisons in Perl.)
--
Andrew (irc:RhodiumToad)
Re: Collation versioning
Oh good. I'd missed that detail. So that eases the RI constraint concern.
(In my previous job, we wanted case/accent insensitive collations, so equal
did not mean binary equal which added a whole extra level of fun.)
On Sun, Sep 16, 2018 at 11:39 AM Andrew Gierth <andrew@tao11.riddles.org.uk>
wrote:
Show quoted text
"Douglas" == Douglas Doole <dougdoole@gmail.com> writes:
Douglas> And constraints problems are even easier than triggers.
Douglas> Consider a database with complex BI rules that are implemented
Douglas> through triggers that fire when values are/are not equal. If
Douglas> the equality of strings change, there could be bad data
Douglas> throughout the tables.Perhaps fortunately, collation changes cannot (in PG) affect the
equality or non-equality of strings (at least of text/varchar/char
types, citext is a different matter). For the builtin string types, PG
follows the rule that if the collation calls the values equal, they are
ordered secondarily in codepoint order; only byte-identical values can
actually be equal (we need this in order for hashing to work in the
absence of a strxfrm implementation that we can trust).(This is the same rule used for string comparisons in Perl.)
--
Andrew (irc:RhodiumToad)
Re: Collation versioning
On Mon, Sep 17, 2018 at 6:13 AM Douglas Doole <dougdoole@gmail.com> wrote:
On Sun, Sep 16, 2018 at 1:20 AM Thomas Munro <thomas.munro@enterprisedb.com> wrote:
3. Fix the tracking of when reindexes need to be rebuilt, so that you
can't get it wrong (as you're alluding to above).I've mentioned this in the past, but didn't seem to get any traction, so I'll try it again ;-)
Hi Doug,
Probably because we agree with you, but don't have all the answers :-)
The focus on indexes when a collation changes is, in my opinion, the least of the problems. You definitely have to worry about indexes, but they can be easily rebuilt. What about other places where collation is hardened into the system, such as constraints?
We have to start somewhere and indexes are the first thing that people
notice, and are much likely to actually be a problem (personally I've
encountered many cases of index corruption due to collation changes in
the wild, but never a constraint corruption, though I fully understand
the theoretical concern). Several of us have observed specifically
that the same problems apply to CHECK constraints and PARTITION
boundaries, and there may be other things like that. You could
imagine tracking collation dependencies on those, requiring a RECHECK
or REPARTITION operation to update them after a depended-on collation
version changes.
Perhaps that suggests that there should be a more general way to store
collation dependencies -- something more like pg_depend, rather than
bolting something like indcollversion onto indexes and every other
kind of catalog that might need it. I don't know.
For example, in ICU 4.6 the handling of accents changed for French. Previously accents were considered right-to-left but ICU 4.6 reversed this. So consider a constraint like CHECK COL < 'coté' (last letter is U+00E9, small letter e with acute). Prior to ICU 4.6 the value 'côte' (second letter is U+00F4, small letter o with circumflex) would have passed this constraint. With 4.6 or later it would be rejected because of the accent ordering change. As soon as the collation changes, this table becomes inconsistent and a reindex isn't going to help it. This becomes a data cleansing problem at this point (which sucks for the user because their data was clean immediately prior to the "upgrade").
Yeah, that's a fun case. I haven't checked recently, but last time I
looked[1]/messages/by-id/CAEepm=30SQpEUjau=dScuNeVZaK2kJ6QQDCHF75u5W=Cz=3Scw@mail.gmail.com and if I understood that byte sequence correctly, they were
still using that whacky right-to-left accent sorting logic for fr_CA,
but had given up on it in fr_FR (though there was still a way to ask
for it). Vive le Québec libre.
...
And constraints problems are even easier than triggers. Consider a database with complex BI rules that are implemented through triggers that fire when values are/are not equal. If the equality of strings change, there could be bad data throughout the tables. (At least with constraints the inter-column dependencies are explicit in the catalogs. With triggers anything goes.)
Once you get into downstream effects of changes (whether they are
recorded in the database or elsewhere), I think it's basically beyond
our event horizon. Why and when did the collation definition change
(bug fix in CLDR, decree by the Académie Française taking effect on 1
January 2019, ...)? We could all use bitemporal databases and
multi-version ICU, but at some point it all starts to look like an
episode of Dr Who. I think we should make a clear distinction between
things that invalidate the correct working of the database, and more
nebulous effects that we can't possibly track in general.
[1]: /messages/by-id/CAEepm=30SQpEUjau=dScuNeVZaK2kJ6QQDCHF75u5W=Cz=3Scw@mail.gmail.com
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
Greetings,
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
On Mon, Sep 17, 2018 at 6:13 AM Douglas Doole <dougdoole@gmail.com> wrote:
On Sun, Sep 16, 2018 at 1:20 AM Thomas Munro <thomas.munro@enterprisedb.com> wrote:
3. Fix the tracking of when reindexes need to be rebuilt, so that you
can't get it wrong (as you're alluding to above).I've mentioned this in the past, but didn't seem to get any traction, so I'll try it again ;-)
Probably because we agree with you, but don't have all the answers :-)
Agreed.
The focus on indexes when a collation changes is, in my opinion, the least of the problems. You definitely have to worry about indexes, but they can be easily rebuilt. What about other places where collation is hardened into the system, such as constraints?
We have to start somewhere and indexes are the first thing that people
notice, and are much likely to actually be a problem (personally I've
encountered many cases of index corruption due to collation changes in
the wild, but never a constraint corruption, though I fully understand
the theoretical concern). Several of us have observed specifically
that the same problems apply to CHECK constraints and PARTITION
boundaries, and there may be other things like that. You could
imagine tracking collation dependencies on those, requiring a RECHECK
or REPARTITION operation to update them after a depended-on collation
version changes.Perhaps that suggests that there should be a more general way to store
collation dependencies -- something more like pg_depend, rather than
bolting something like indcollversion onto indexes and every other
kind of catalog that might need it. I don't know.
Agreed. If we start thinking about pg_depend then maybe we realize
that this all comes back to pg_attribute as the holder of the
column-level information and maybe what we should be thinking about is a
way to encode version information into the typmod for text-based
types...
And constraints problems are even easier than triggers. Consider a database with complex BI rules that are implemented through triggers that fire when values are/are not equal. If the equality of strings change, there could be bad data throughout the tables. (At least with constraints the inter-column dependencies are explicit in the catalogs. With triggers anything goes.)
Once you get into downstream effects of changes (whether they are
recorded in the database or elsewhere), I think it's basically beyond
our event horizon. Why and when did the collation definition change
(bug fix in CLDR, decree by the Académie Française taking effect on 1
January 2019, ...)? We could all use bitemporal databases and
multi-version ICU, but at some point it all starts to look like an
episode of Dr Who. I think we should make a clear distinction between
things that invalidate the correct working of the database, and more
nebulous effects that we can't possibly track in general.
I tend to agree in general, but I don't think it's beyond us to consider
multi-version ICU and being able to perform online reindexing (such that
a given system could be migrated from one collation to another over a
time while the system is still online, instead of having to take a
potentially long downtime hit to rebuild indexes after an upgrade, or
having to rebuild the entire system using some kind of logical
replication...).
Thanks!
Stephen
Re: Collation versioning
On Mon, Sep 17, 2018 at 9:02 AM Stephen Frost <sfrost@snowman.net> wrote:
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
Once you get into downstream effects of changes (whether they are
recorded in the database or elsewhere), I think it's basically beyond
our event horizon. Why and when did the collation definition change
(bug fix in CLDR, decree by the Académie Française taking effect on 1
January 2019, ...)? We could all use bitemporal databases and
multi-version ICU, but at some point it all starts to look like an
episode of Dr Who. I think we should make a clear distinction between
things that invalidate the correct working of the database, and more
nebulous effects that we can't possibly track in general.I tend to agree in general, but I don't think it's beyond us to consider
multi-version ICU and being able to perform online reindexing (such that
a given system could be migrated from one collation to another over a
time while the system is still online, instead of having to take a
potentially long downtime hit to rebuild indexes after an upgrade, or
having to rebuild the entire system using some kind of logical
replication...).
It's a very interesting idea with a high nerd-sniping factor[1]https://xkcd.com/356/.
Practically speaking, I wonder if you can actually do that with
typical Linux distributions where the ICU data is in a shared library
(eg libicudata.so.57), and may also be dependent on the ICU code
version (?) -- do you run into problems linking to several of them at
the same time? Maybe you have to ship your own ICU collations in
"data" form to pull that off. But someone mentioned that
distributions don't like you to do that (likewise for tzinfo and other
such things that no one wants 42 copies of on their system).
Actually, if I had infinite resources I'd really like to go and make
libc support multiple collation versions with a standard interface
(technically easy, bureaucratically hard); I don't really like leaving
libc behind. But I digress.
I'd like to propose the 3 more humble goals I mentioned a few messages
back as earlier steps. OS collation changes aren't really like Monty
Python's Spanish Inquisition: they usually hit you when you're doing
major operating system upgrades or setting up a streaming replica to a
different OS version IIUC. That is, they probably happen during
maintenance windows when REINDEX would hopefully be plausible, and
presumably critical systems get tested on the new OS version before
production is upgraded. It'd be kind to our users to make the problem
non-silent at that time so they can plan for it (and of course also
alert them if it happens when nobody expects it, because knowing you
have a problem is better than not knowing).
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
Greetings,
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
I'd like to propose the 3 more humble goals I mentioned a few messages
back as earlier steps. OS collation changes aren't really like Monty
Python's Spanish Inquisition: they usually hit you when you're doing
major operating system upgrades or setting up a streaming replica to a
different OS version IIUC. That is, they probably happen during
maintenance windows when REINDEX would hopefully be plausible, and
presumably critical systems get tested on the new OS version before
production is upgraded. It'd be kind to our users to make the problem
non-silent at that time so they can plan for it (and of course also
alert them if it happens when nobody expects it, because knowing you
have a problem is better than not knowing).
Just to be clear, I'm all for this, but wanted to bring up the farther
out goal to make sure we're thinking about how to eventually get there
from here- and to make sure we aren't making it more difficult to get
there with the proposed catalog changes for these shorter-term goals.
Thanks!
Stephen
Re: Collation versioning
On Sun, Sep 16, 2018 at 1:14 PM Thomas Munro <thomas.munro@enterprisedb.com>
wrote:
We have to start somewhere and indexes are the first thing that people
notice, and are much likely to actually be a problem (personally I've
encountered many cases of index corruption due to collation changes in
the wild, but never a constraint corruption,
Problems that people notice are good - it's the ones that they don't notice
that cause real grief ;-)
Given that PostgreSQL requires equal values to be binary identical I think
that you'll avoid a lot of the problems that caused me so much trouble in
DB2. Even if someone creates a range constraint or partitioned table, the
boundary values aren't typically going to be impacted (my French example
was a little contrived). That said, there have been a few changes that
would have much more obvious impacts. One language stopped treating 'ch' as
a single collating element that came somewhere after 'c'+'z' and treated it
as 'c'+'h' instead. And and one of the North Germanic languages moved an
accented character from after 'z' to before 'z' (or maybe it was vice versa
- I miss my library of Unicode presentations).
Once you get into downstream effects of changes (whether they are
recorded in the database or elsewhere), I think it's basically beyond
our event horizon.
...
I think we should make a clear distinction between
things that invalidate the correct working of the database, and more
nebulous effects that we can't possibly track in general.
I agree that PostgreSQL can't resolve the downstream changes, but that's a
very subtle distinction. As a user, if an upgrade caused my data to no
longer comply with my carefully architected and database enforced BI rules,
I would definitely argue that the correct working of the database has been
invalidated. (You can make technical arguments about the OS upgrading the
library, but the fundamental issue is that my PostgreSQL database is
broken.)
You can probably argue that PostgreSQL and DB2 users look at the world
differently, but that's why DB2 ended up shipping its own copies of the ICU
library. Once a user creates an object using ICU vX, we made sure that
version of the library was always available to avoid these problems. (The
libraries were on a private path with non-standard names so there was no
collision with the OS library. Fortunately I'd moved on before anyone
really started complaining about why 5 copies of ICU were being installed
when they weren't even using Unicode.)
[1]
/messages/by-id/CAEepm=30SQpEUjau=dScuNeVZaK2kJ6QQDCHF75u5W=Cz=3Scw@mail.gmail.com
I'd missed this post. In it you asked:
I wonder how it manages to deal with fr_CA's reversed secondary weighting
rule which requires you to consider diacritics in reverse order --
apparently abandoned in France but still used in Canada -- using a fixed
size space for state between calls.
ucol_nextSortKeyPart() only keeps track of the current position in the
generated sort key as its state. So, if you call it multiple times to
generate the sort key piecemeal, it recomputes the entire sort key until it
has enough bytes to satisfy your request. (That is, if you're doing 4 bytes
at a time, on the first call it generates and returns bytes 1-4. On the
second call it generates 1-8 and returns 5-8. Next it generates 1-12 and
returns 9-12.) Needless to say, this gets very expensive very fast.
Re: Collation versioning
On Mon 17 Sep 2018, 13:02 Douglas Doole, <dougdoole@gmail.com> wrote:
On Sun, Sep 16, 2018 at 1:14 PM Thomas Munro <
thomas.munro@enterprisedb.com> wrote:We have to start somewhere and indexes are the first thing that people
notice, and are much likely to actually be a problem (personally I've
encountered many cases of index corruption due to collation changes in
the wild, but never a constraint corruption,Problems that people notice are good - it's the ones that they don't
notice that cause real grief ;-)
...Once you get into downstream effects of changes (whether they are
recorded in the database or elsewhere), I think it's basically beyond
our event horizon.
...
I think we should make a clear distinction between
things that invalidate the correct working of the database, and more
nebulous effects that we can't possibly track in general.I agree that PostgreSQL can't resolve the downstream changes, but that's a
very subtle distinction. As a user, if an upgrade caused my data to no
longer comply with my carefully architected and database enforced BI rules,
I would definitely argue that the correct working of the database has been
invalidated. (You can make technical arguments about the OS upgrading the
library, but the fundamental issue is that my PostgreSQL database is
broken.)
Well things like partition exclusion and even join elimination depend on
constraints being consistent so I don't think it's as easy to write off as
that either.
But it's true that we have to start somewhere and collation changes are
much more likely to be spotted in indexes and cause much more visible
issues.
You can probably argue that PostgreSQL and DB2 users look at the world
differently, but that's why DB2 ended up shipping its own copies of the ICU
library. Once a user creates an object using ICU vX, we made sure that
version of the library was always available to avoid these problems.
This seems like a terrible idea in the open source world. Surely collation
versioning means new ICU libraries can still provide the old collation
rules so even if you update the library you can request the old version? We
shouldn't need that actual old code with all its security holes and bugs
just to get the old collation version.
Re: Collation versioning
On Mon, Sep 17, 2018 at 12:32 PM Greg Stark <stark@mit.edu> wrote:
This seems like a terrible idea in the open source world. Surely collation
versioning means new ICU libraries can still provide the old collation
rules so even if you update the library you can request the old version? We
shouldn't need that actual old code with all its security holes and bugs
just to get the old collation version.
We asked long and hard for this feature from the ICU team but they kept
arguing it was too hard to do. There are apparently some tight couplings
between the code and each version of CLDR. So the only way to support old
collations is to ship the entire old library. (They even added make rules
to allow the entire API to be version extended to accommodate this
requirement.)
Even bug fixes are potentially problematic because the fix may alter how
some code points collate. The ICU team won't (or at least wouldn't - been a
few years since I dealt with them) guarantee any sort of backwards
compatibility between code drops.
As an aside, they did look at making the CLDR data a separate data file
that could be read by any version of the code (before finding there were
too many dependencies). One thing that they discovered is that this
approach didn't save much disk since the CLDR data is something like 90-95%
of the library. So while it would have made the calling code a lot cleaner,
it wasn't the huge footprint win we'd been hoping for.
Re: Collation versioning
On Mon, Sep 17, 2018 at 9:02 AM Stephen Frost <sfrost@snowman.net> wrote:
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
On Mon, Sep 17, 2018 at 6:13 AM Douglas Doole <dougdoole@gmail.com> wrote:
On Sun, Sep 16, 2018 at 1:20 AM Thomas Munro <thomas.munro@enterprisedb.com> wrote:
3. Fix the tracking of when reindexes need to be rebuilt, so that you
can't get it wrong (as you're alluding to above).I've mentioned this in the past, but didn't seem to get any traction, so I'll try it again ;-)
Probably because we agree with you, but don't have all the answers :-)
Agreed.
The focus on indexes when a collation changes is, in my opinion, the least of the problems. You definitely have to worry about indexes, but they can be easily rebuilt. What about other places where collation is hardened into the system, such as constraints?
We have to start somewhere and indexes are the first thing that people
notice, and are much likely to actually be a problem (personally I've
encountered many cases of index corruption due to collation changes in
the wild, but never a constraint corruption, though I fully understand
the theoretical concern). Several of us have observed specifically
that the same problems apply to CHECK constraints and PARTITION
boundaries, and there may be other things like that. You could
imagine tracking collation dependencies on those, requiring a RECHECK
or REPARTITION operation to update them after a depended-on collation
version changes.Perhaps that suggests that there should be a more general way to store
collation dependencies -- something more like pg_depend, rather than
bolting something like indcollversion onto indexes and every other
kind of catalog that might need it. I don't know.Agreed. If we start thinking about pg_depend then maybe we realize
that this all comes back to pg_attribute as the holder of the
column-level information and maybe what we should be thinking about is a
way to encode version information into the typmod for text-based
types...
So to be more concrete: pg_depend could have a new column
"refobjversion". Whenever indexes are created or rebuilt, we'd
capture the current version string in the pg_depend rows that link
index attributes and collations. Then we'd compare those against the
current value when we first open an index and complain if they don't
match. (In this model there would be no "collversion" column in the
pg_collation catalog.)
That'd leave a place for other kinds of database objects (CHECKs,
PARTITIONS, ...) to store their version dependency, if someone later
wants to add support for that.
I'm not sure if my idea about updating the default collation row in
newly created databases has legs though. Any thoughts on that?
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
Greetings,
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
On Mon, Sep 17, 2018 at 9:02 AM Stephen Frost <sfrost@snowman.net> wrote:
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
On Mon, Sep 17, 2018 at 6:13 AM Douglas Doole <dougdoole@gmail.com> wrote:
On Sun, Sep 16, 2018 at 1:20 AM Thomas Munro <thomas.munro@enterprisedb.com> wrote:
3. Fix the tracking of when reindexes need to be rebuilt, so that you
can't get it wrong (as you're alluding to above).I've mentioned this in the past, but didn't seem to get any traction, so I'll try it again ;-)
Probably because we agree with you, but don't have all the answers :-)
Agreed.
The focus on indexes when a collation changes is, in my opinion, the least of the problems. You definitely have to worry about indexes, but they can be easily rebuilt. What about other places where collation is hardened into the system, such as constraints?
We have to start somewhere and indexes are the first thing that people
notice, and are much likely to actually be a problem (personally I've
encountered many cases of index corruption due to collation changes in
the wild, but never a constraint corruption, though I fully understand
the theoretical concern). Several of us have observed specifically
that the same problems apply to CHECK constraints and PARTITION
boundaries, and there may be other things like that. You could
imagine tracking collation dependencies on those, requiring a RECHECK
or REPARTITION operation to update them after a depended-on collation
version changes.Perhaps that suggests that there should be a more general way to store
collation dependencies -- something more like pg_depend, rather than
bolting something like indcollversion onto indexes and every other
kind of catalog that might need it. I don't know.Agreed. If we start thinking about pg_depend then maybe we realize
that this all comes back to pg_attribute as the holder of the
column-level information and maybe what we should be thinking about is a
way to encode version information into the typmod for text-based
types...So to be more concrete: pg_depend could have a new column
"refobjversion". Whenever indexes are created or rebuilt, we'd
capture the current version string in the pg_depend rows that link
index attributes and collations. Then we'd compare those against the
current value when we first open an index and complain if they don't
match. (In this model there would be no "collversion" column in the
pg_collation catalog.)
I'm really not sure why you're pushing to have this in pg_depend..
That'd leave a place for other kinds of database objects (CHECKs,
PARTITIONS, ...) to store their version dependency, if someone later
wants to add support for that.
Isn't what matters here where the data's stored, as in, in a column..?
All of those would already have dependencies on the column so that they
can be tracked back there.
I'm not sure if my idea about updating the default collation row in
newly created databases has legs though. Any thoughts on that?
My initial reaction is that we should have a version included basically
everywhere and then let users decide how they want to change it. For a
new cluster, I'd agree with using the latest available (while allowing
it to be chosen if a user wishes for something else) but I'm not sure
I'd go farther than that.
Thanks!
Stephen
Re: Collation versioning
On Wed, Sep 19, 2018 at 12:48 AM Stephen Frost <sfrost@snowman.net> wrote:
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
So to be more concrete: pg_depend could have a new column
"refobjversion". Whenever indexes are created or rebuilt, we'd
capture the current version string in the pg_depend rows that link
index attributes and collations. Then we'd compare those against the
current value when we first open an index and complain if they don't
match. (In this model there would be no "collversion" column in the
pg_collation catalog.)I'm really not sure why you're pushing to have this in pg_depend..
That'd leave a place for other kinds of database objects (CHECKs,
PARTITIONS, ...) to store their version dependency, if someone later
wants to add support for that.Isn't what matters here where the data's stored, as in, in a column..?
All of those would already have dependencies on the column so that they
can be tracked back there.
Suppose I have a table "emp" and indexes "emp_firstname_idx" and
"emp_lastname_idx". Suppose I created them in a sequence like this:
0: collation fr_CA has version "30"
1: create table emp (firstname text collate "fr_CA", lastname text
collate "fr_CA");
2: create index on emp(firstname);
3: [upgrade operating system]; now collation fr_CA has version "31"
4: create index on emp(lastname);
Now I have two indexes, built when different versions of the collation
were in effect. One of them is potentially corrupted, the other
isn't. Where are you going to record that? Earlier I suggested that
pg_index could have an indcollversion column, so that
emp_firstname_idx's row would hold {"30"} and emp_lastname_idx's row
would hold {"31"}. It would be captured at CREATE INDEX time, and
after that the only way to change it would be to REINDEX, and whenever
it disagrees with the current version according to the provider you'd
get a warning that you can only clear by running REINDEX. Then I
suggested that perhaps pg_depend might be a better place for it,
because it would generalise to other kinds of object too.
For example, suppose I create a constraint CHECK (foo < 'côté') [evil
laugh]. The pg_depend row that links the constraint and the collation
could record the current version as of the moment the constraint was
defined. After an OS upgrade that changes the reported version, I'd
see a warning whenever loading the check constraint, and the only way
to clear it would be to drop and recreate the constraint. (I'm not
proposing we do that, just trying to demonstrate that pg_depend might
be a tidier and more general solution than adding 'collation version'
columns holding arrays of version strings to multiple catalogs. So
help me Codd.)
Just an idea...
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
Greetings,
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
On Wed, Sep 19, 2018 at 12:48 AM Stephen Frost <sfrost@snowman.net> wrote:
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
So to be more concrete: pg_depend could have a new column
"refobjversion". Whenever indexes are created or rebuilt, we'd
capture the current version string in the pg_depend rows that link
index attributes and collations. Then we'd compare those against the
current value when we first open an index and complain if they don't
match. (In this model there would be no "collversion" column in the
pg_collation catalog.)I'm really not sure why you're pushing to have this in pg_depend..
That'd leave a place for other kinds of database objects (CHECKs,
PARTITIONS, ...) to store their version dependency, if someone later
wants to add support for that.Isn't what matters here where the data's stored, as in, in a column..?
All of those would already have dependencies on the column so that they
can be tracked back there.Suppose I have a table "emp" and indexes "emp_firstname_idx" and
"emp_lastname_idx". Suppose I created them in a sequence like this:0: collation fr_CA has version "30"
1: create table emp (firstname text collate "fr_CA", lastname text
collate "fr_CA");
2: create index on emp(firstname);
3: [upgrade operating system]; now collation fr_CA has version "31"
4: create index on emp(lastname);Now I have two indexes, built when different versions of the collation
were in effect. One of them is potentially corrupted, the other
isn't. Where are you going to record that? Earlier I suggested that
pg_index could have an indcollversion column, so that
emp_firstname_idx's row would hold {"30"} and emp_lastname_idx's row
would hold {"31"}. It would be captured at CREATE INDEX time, and
after that the only way to change it would be to REINDEX, and whenever
it disagrees with the current version according to the provider you'd
get a warning that you can only clear by running REINDEX. Then I
suggested that perhaps pg_depend might be a better place for it,
because it would generalise to other kinds of object too.
For indexes, just like for tables, we have entries in pg_attribute where
that information would go.
For example, suppose I create a constraint CHECK (foo < 'côté') [evil
laugh]. The pg_depend row that links the constraint and the collation
could record the current version as of the moment the constraint was
defined. After an OS upgrade that changes the reported version, I'd
see a warning whenever loading the check constraint, and the only way
to clear it would be to drop and recreate the constraint. (I'm not
proposing we do that, just trying to demonstrate that pg_depend might
be a tidier and more general solution than adding 'collation version'
columns holding arrays of version strings to multiple catalogs. So
help me Codd.)
The CHECK constraint doesn't need to directly track that information-
it should have a dependency on the column in the table and that's where
the information would be recorded about the current collation version.
Maybe I'm missing something but I have to admit that I feel like
pg_depend is being looked at here because everything goes through it-
but everything goes through it because it's simple and we just use it to
get to other things that have the complete definition of the object.
Lots and lots of things in pg_depend would have zero use for such a
field and I'm a bit worried you'd possibly also get into cases where
you've got different collation versions for the same object because of
the different dependencies into it... even if you don't, you're
duplicating that information into every dependency, aren't you?
Thanks!
Stephen
Re: Collation versioning
The CHECK constraint doesn't need to directly track that information-
it should have a dependency on the column in the table and that's where
the information would be recorded about the current collation version.
Just to have fun throwing odd cases out, how would something like this be
recorded?
Database default collation: en_US
CREATE TABLE t (c1 TEXT, c2 TEXT, c3 TEXT,
CHECK (c1 COLLATE "fr_FR" BETWEEN c2 COLLATE "fr_FR" AND c3 COLLATE
"fr_FR"));
You could even be really warped and apply multiple collations on a single
column in a single constraint.
Re: Collation versioning
Greetings,
* Douglas Doole (dougdoole@gmail.com) wrote:
The CHECK constraint doesn't need to directly track that information-
it should have a dependency on the column in the table and that's where
the information would be recorded about the current collation version.Just to have fun throwing odd cases out, how would something like this be
recorded?Database default collation: en_US
CREATE TABLE t (c1 TEXT, c2 TEXT, c3 TEXT,
CHECK (c1 COLLATE "fr_FR" BETWEEN c2 COLLATE "fr_FR" AND c3 COLLATE
"fr_FR"));You could even be really warped and apply multiple collations on a single
column in a single constraint.
Once it gets to an expression and not just a simple check, I'd think
we'd record it in the expression..
Thanks!
Stephen
Re: Collation versioning
On Wed, Sep 19, 2018 at 10:09 AM Stephen Frost <sfrost@snowman.net> wrote:
* Douglas Doole (dougdoole@gmail.com) wrote:
The CHECK constraint doesn't need to directly track that information-
it should have a dependency on the column in the table and that's where
the information would be recorded about the current collation version.Just to have fun throwing odd cases out, how would something like this be
recorded?Database default collation: en_US
CREATE TABLE t (c1 TEXT, c2 TEXT, c3 TEXT,
CHECK (c1 COLLATE "fr_FR" BETWEEN c2 COLLATE "fr_FR" AND c3 COLLATE
"fr_FR"));You could even be really warped and apply multiple collations on a single
column in a single constraint.Once it gets to an expression and not just a simple check, I'd think
we'd record it in the expression..
Maybe I misunderstood, but I don't think it makes sense to have a
collation version "on the column in the table", because (1) that fails
to capture the fact that two CHECK constraints that were defined at
different times might have become dependent on two different versions
(you created one constraint before upgrading and the other after, now
the older one is invalidated and sounds the alarm but the second one
is fine), and (2) the table itself doesn't care about collation
versions since heap tables are unordered; there is no particular
operation on the table that would be the correct time to update the
collation version on a table/column. What we're trying to track is
when objects that in some way depend on the version become
invalidated, so wherever we store it there's going to have to be a
version recorded per dependent object at its creation time, so that's
either new columns on every interested catalog table, or ...
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
Greetings,
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
On Wed, Sep 19, 2018 at 10:09 AM Stephen Frost <sfrost@snowman.net> wrote:
* Douglas Doole (dougdoole@gmail.com) wrote:
The CHECK constraint doesn't need to directly track that information-
it should have a dependency on the column in the table and that's where
the information would be recorded about the current collation version.Just to have fun throwing odd cases out, how would something like this be
recorded?Database default collation: en_US
CREATE TABLE t (c1 TEXT, c2 TEXT, c3 TEXT,
CHECK (c1 COLLATE "fr_FR" BETWEEN c2 COLLATE "fr_FR" AND c3 COLLATE
"fr_FR"));You could even be really warped and apply multiple collations on a single
column in a single constraint.Once it gets to an expression and not just a simple check, I'd think
we'd record it in the expression..Maybe I misunderstood, but I don't think it makes sense to have a
collation version "on the column in the table", because (1) that fails
to capture the fact that two CHECK constraints that were defined at
different times might have become dependent on two different versions
(you created one constraint before upgrading and the other after, now
the older one is invalidated and sounds the alarm but the second one
is fine), and (2) the table itself doesn't care about collation
versions since heap tables are unordered; there is no particular
operation on the table that would be the correct time to update the
collation version on a table/column. What we're trying to track is
when objects that in some way depend on the version become
invalidated, so wherever we store it there's going to have to be a
version recorded per dependent object at its creation time, so that's
either new columns on every interested catalog table, or ...
Today, we work out what operators to use for a given column based on the
data type. This is why I was trying to get at the idea of using typmod
earlier, but if we can't make that work then I'm inclined to try and
figure out a way to get it as close as possible to being associated with
the type.
Yes, the heap is unordered, but the type *is* ordered and we track that
with the type system. Maybe we just extend that somehow, rather than
using the typmod or adding columns into catalogs where we store type
information (such as pg_attribute). Perhaps typmod could be made larger
to be able to support this, that might be another approach.
No where in this does it strike me that it makes sense to push this into
pg_depend though.
Thanks!
Stephen
Re: Collation versioning
On Wed, Sep 19, 2018 at 1:16 PM Stephen Frost <sfrost@snowman.net> wrote:
* Thomas Munro (thomas.munro@enterprisedb.com) wrote:
On Wed, Sep 19, 2018 at 10:09 AM Stephen Frost <sfrost@snowman.net> wrote:
* Douglas Doole (dougdoole@gmail.com) wrote:
The CHECK constraint doesn't need to directly track that information-
it should have a dependency on the column in the table and that's where
the information would be recorded about the current collation version.Just to have fun throwing odd cases out, how would something like this be
recorded?Database default collation: en_US
CREATE TABLE t (c1 TEXT, c2 TEXT, c3 TEXT,
CHECK (c1 COLLATE "fr_FR" BETWEEN c2 COLLATE "fr_FR" AND c3 COLLATE
"fr_FR"));You could even be really warped and apply multiple collations on a single
column in a single constraint.Once it gets to an expression and not just a simple check, I'd think
we'd record it in the expression..Maybe I misunderstood, but I don't think it makes sense to have a
collation version "on the column in the table", because (1) that fails
to capture the fact that two CHECK constraints that were defined at
different times might have become dependent on two different versions
(you created one constraint before upgrading and the other after, now
the older one is invalidated and sounds the alarm but the second one
is fine), and (2) the table itself doesn't care about collation
versions since heap tables are unordered; there is no particular
operation on the table that would be the correct time to update the
collation version on a table/column. What we're trying to track is
when objects that in some way depend on the version become
invalidated, so wherever we store it there's going to have to be a
version recorded per dependent object at its creation time, so that's
either new columns on every interested catalog table, or ...Today, we work out what operators to use for a given column based on the
data type. This is why I was trying to get at the idea of using typmod
earlier, but if we can't make that work then I'm inclined to try and
figure out a way to get it as close as possible to being associated with
the type.Yes, the heap is unordered, but the type *is* ordered and we track that
with the type system. Maybe we just extend that somehow, rather than
using the typmod or adding columns into catalogs where we store type
information (such as pg_attribute). Perhaps typmod could be made larger
to be able to support this, that might be another approach.
Can you show how this would look in the catalogs, for Doug's example
above? There is no pg_attribute or similar that applies here.
Perhaps you want to bury the versions inside the serialised expression
tree? Or store it in a pair of arrays on the pg_constraint row?
No where in this does it strike me that it makes sense to push this into
pg_depend though.
In my scheme, the pg_depend row that already exists to record the
dependency between the pg_constraint row and the pg_collation row for
"fr_FR" holds a refobjversion value captured at constraint creation
time. Maybe that's a bit strange, but the alternatives I have thought
of so far seemed ad hoc and strange too.
Here's a Sunday afternoon rapid prototype of the concept, implemented
for indexes and check constraints. Examples of the output you get
when you access those database objects for the first time in each
backend:
WARNING: index "foo_i_idx" depends on collation 16385 version
"30.0.1", but the current version is "30.0.2"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
WARNING: constraint "t_i_check" depends on collation 12018 version
"30.0.1", but the current version is "30.0.2"
DETAIL: The constraint may be corrupted due to changes in sort order.
HINT: Drop and recreate the constraint to avoid the risk of corruption.
I didn't try to tackle the default collation yet, so to try this you
need to use non-default collations.
Whether we'd actually want to do this for CHECK constraints or any of
this, I don't know. It's theoretically the right thing to do, but
most people probably don't want to be nagged to recreate (or recheck
in some new way) all their constraints every time they do an OS
upgrade, and in fact there is almost never anything actually wrong if
they don't... unless they're unlucky. But I wanted to do this to show
that it's a fairly general way to deal with dependencies between
database objects and collation versions using the existing links
between them in a relational sort of way. I guess you might want a
command or tool to find all such problems proactively and recheck,
reindex, repartition etc as required to make it shut up, and that
should be quite easy to drive from this design.
--
Thomas Munro
http://www.enterprisedb.com
Attachments:
0001-Remove-pg_collation.collversion.patchapplication/octet-stream; name=0001-Remove-pg_collation.collversion.patchDownload
From 4b60c1d7152e67f6ef304cce548b2e8ab8f82ad8 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Sun, 23 Sep 2018 09:13:48 +1200
Subject: [PATCH 1/6] Remove pg_collation.collversion.
A later patch will add version tracking to individual indexes.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ----
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 53 -----------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 84 ---------------------------
src/backend/nodes/copyfuncs.c | 13 -----
src/backend/nodes/equalfuncs.c | 11 ----
src/backend/parser/gram.y | 18 +-----
src/backend/tcop/utility.c | 12 ----
src/backend/utils/adt/pg_locale.c | 37 ------------
src/include/catalog/pg_collation.dat | 7 +--
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ----
15 files changed, 5 insertions(+), 269 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 0179deea2e0..401ff5ee9bd 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c44417d868d..08ed9255b3d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -20039,10 +20039,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index b51b3a25647..4241ec9f5a3 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,62 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index ce7e5fb5cc1..567eb786065 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -48,7 +48,6 @@ CollationCreate(const char *collname, Oid collnamespace,
char collprovider,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -162,10 +161,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 8fb51e8c3d1..832fce92631 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -65,7 +65,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
char *collproviderstr = NULL;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -157,9 +156,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (providerEl)
collproviderstr = defGetString(providerEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -196,9 +192,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -206,7 +199,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -257,80 +249,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = heap_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- heap_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -588,7 +506,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -649,7 +566,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7c8220cf651..20e64e0eaf9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3164,16 +3164,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5175,9 +5165,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 378f2facb84..13eacd8e988 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1097,14 +1097,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3234,9 +3226,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4bd2223f267..3ea45c92ea1 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -244,7 +244,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -820,7 +820,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10229,21 +10228,6 @@ DropdbStmt: DROP DATABASE database_name
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b5804f64ad4..0740bbf1ac5 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1667,10 +1667,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = CreateStatistics((CreateStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2768,10 +2764,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3372,10 +3364,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index a3dc3be5a87..ddfe85e5940 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1294,8 +1294,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1393,41 +1391,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ad16116cf4d..b061a6f602e 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 7e0f4461c61..e671f65ac9f 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -35,10 +35,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -62,7 +58,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
char collprovider,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index f259890e43c..286ed4f6122 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -48,7 +48,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 9b0f00a997e..3cfc35f2175 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 62209a8f102..1ccda6a963c 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1817,17 +1817,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
--
2.17.0
0002-Add-pg_depend.refobjversion.patchapplication/octet-stream; name=0002-Add-pg_depend.refobjversion.patchDownload
From 34fb46e8e6f8d74aef5fecfad66033943a0d08dc Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Sun, 23 Sep 2018 09:39:16 +1200
Subject: [PATCH 2/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 22 ++++++++++--
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 5 +++
src/include/catalog/pg_depend.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
6 files changed, 57 insertions(+), 30 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 4f1d3653575..67f58517a70 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1376,7 +1376,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1459,7 +1460,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies */
if (!ignore_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
free_object_addresses(self_addrs);
@@ -1467,7 +1469,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2390,7 +2393,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 2ea05f350b3..4caea6cbe6f 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -45,7 +45,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -55,6 +70,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -102,9 +118,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ values[Anum_pg_depend_refobjversion - 1] = version ? NameGetDatum(version) : CStringGetDatum("");
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
@@ -115,6 +131,8 @@ recordMultipleDependencies(const ObjectAddress *depender,
heap_freetuple(tup);
}
+ if (version)
+ ++version;
}
if (indstate != NULL)
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index cb8c7450d9b..3c3a23d7871 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1604,55 +1604,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c6..6ae4932cfba 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -233,8 +233,13 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced, const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index 482b8bd2516..3fb317b3aea 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -55,6 +55,7 @@ CATALOG(pg_depend,2608,DependRelationId) BKI_WITHOUT_OIDS
Oid refclassid; /* OID of table containing object */
Oid refobjid; /* OID of object itself */
int32 refobjsubid; /* column number, or 0 if not used */
+ NameData refobjversion; /* version tracking, or empty if not used */
/*
* Precise semantics of the relationship are specified by the deptype
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 2d3522b500d..26a72f59aa6 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | refobjversion | deptype
+---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
-- **************** pg_shdepend ****************
--
2.17.0
0003-Track-collation-versions-for-indexes.patchapplication/octet-stream; name=0003-Track-collation-versions-for-indexes.patchDownload
From c566463ced067650b2f03ef260d7b3144f452ae2 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Sun, 23 Sep 2018 14:43:01 +1200
Subject: [PATCH 3/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. Whenever we load an index into the
relcache, check if the collation versions still match those reported
by the collation provider. Warn that the index may be corrupted if
not.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 57 ++++++++++++++++++++++
src/backend/catalog/index.c | 78 +++++++++++++++++++++++++++++-
src/backend/utils/adt/pg_locale.c | 21 ++++++++
src/backend/utils/cache/relcache.c | 4 ++
src/include/catalog/dependency.h | 8 +++
src/include/catalog/index.h | 2 +
src/include/utils/pg_locale.h | 1 +
7 files changed, 170 insertions(+), 1 deletion(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 67f58517a70..c674120eaf5 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -422,6 +422,63 @@ performMultipleDeletions(const ObjectAddresses *objects,
heap_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = heap_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ NameData *new_version;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ new_version = callback(&otherObject, &foundDep->refobjversion,
+ userdata);
+ if (new_version)
+ {
+ /* Make a modifyable copy. */
+ tup = heap_copytuple(tup);
+ foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ foundDep->refobjversion = *new_version;
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ heap_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7eb3e351667..96a8facfb29 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -69,6 +69,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/syscache.h"
@@ -1099,11 +1100,15 @@ index_create(Relation heapRelation,
if (OidIsValid(collationObjectId[i]) &&
collationObjectId[i] != DEFAULT_COLLATION_OID)
{
+ NameData version;
+
referenced.classId = CollationRelationId;
referenced.objectId = collationObjectId[i];
referenced.objectSubId = 0;
+ get_collation_version_for_oid(referenced.objectId, &version);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnVersion(&myself, &referenced, &version,
+ DEPENDENCY_NORMAL);
}
}
@@ -1207,6 +1212,74 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static NameData *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ NameData current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+ if (strncmp(NameStr(*version),
+ NameStr(current_version),
+ sizeof(NameData)) != 0)
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation %u version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ otherObject->objectId,
+ NameStr(*version),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static NameData *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NameData *current_version = (NameData *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ get_collation_version_for_oid(otherObject->objectId, current_version);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_constraint_create
*
@@ -3803,6 +3876,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
heap_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index ddfe85e5940..da651344e14 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1443,6 +1443,27 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collatoin OID.
+ */
+void
+get_collation_version_for_oid(Oid oid, NameData *output)
+{
+ HeapTuple tp;
+ Form_pg_collation collform;
+ const char *version;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ memset(output, 0, sizeof(NameData));
+ if (version)
+ strncpy(NameStr(*output), version, sizeof(NameData));
+ ReleaseSysCache(tp);
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a4fc0011031..b58d4f2af6d 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -1452,6 +1452,10 @@ RelationInitIndexAccessInfo(Relation relation)
indcoll = (oidvector *) DatumGetPointer(indcollDatum);
memcpy(relation->rd_indcollation, indcoll->values, indnkeyatts * sizeof(Oid));
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsCatalogRelation(relation))
+ index_check_collation_versions(RelationGetRelid(relation));
+
/*
* indclass cannot be referenced directly through the C struct, because it
* comes after the variable-width indkey field. Must extract the datum
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 6ae4932cfba..66befde7568 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -227,6 +227,14 @@ extern void record_object_address_dependencies(const ObjectAddress *depender,
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef NameData *(*VisitDependentObjectsFun)(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f20c5f789b1..e232dccaef0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -103,6 +103,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 88a3134862f..9c05374c32c 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,6 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern void get_collation_version_for_oid(Oid collid, NameData *output);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
--
2.17.0
0004-Track-collation-versions-for-CHECK-constraints.patchapplication/octet-stream; name=0004-Track-collation-versions-for-CHECK-constraints.patchDownload
From d85668b2ea0605cbfd20342a852a04057f76326b Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Mon, 24 Sep 2018 00:20:28 +1200
Subject: [PATCH 4/6] Track collation versions for CHECK constraints.
Record the current version of dependent collations in pg_depend when
creating CHECK constraints, and then warn if the collation versions
change.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 28 +++++++++++++++++--
src/backend/catalog/pg_constraint.c | 43 +++++++++++++++++++++++++++++
src/backend/utils/cache/relcache.c | 4 +++
src/include/catalog/pg_constraint.h | 3 ++
4 files changed, 76 insertions(+), 2 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index c674120eaf5..f924dbaf242 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/syscache.h"
#include "utils/tqual.h"
@@ -1399,6 +1400,21 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+static NameData *
+capture_collation_versions(ObjectAddresses *addrs)
+{
+ NameData *results = palloc0(sizeof(NameData) * addrs->numrefs);
+
+ for (int i = 0; i < addrs->numrefs; ++i)
+ {
+ if (addrs->refs[i].classId != CollationRelationId)
+ continue;
+ get_collation_version_for_oid(addrs->refs[i].objectId, &results[i]);
+ }
+
+ return results;
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1418,6 +1434,7 @@ recordDependencyOnExpr(const ObjectAddress *depender,
Node *expr, List *rtable,
DependencyType behavior)
{
+ NameData *versions;
find_expr_references_context context;
context.addrs = new_object_addresses();
@@ -1431,9 +1448,12 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* Remove any duplicates */
eliminate_duplicate_dependencies(context.addrs);
+ versions = capture_collation_versions(context.addrs);
+
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
+ versions,
context.addrs->numrefs,
behavior);
@@ -1464,6 +1484,7 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
{
find_expr_references_context context;
RangeTblEntry rte;
+ NameData *versions;
context.addrs = new_object_addresses();
@@ -1524,9 +1545,12 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
free_object_addresses(self_addrs);
}
+ versions = capture_collation_versions(context.addrs);
+
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
+ versions,
context.addrs->numrefs,
behavior);
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6781b00c6e6..2972706cf4c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -23,6 +23,7 @@
#include "catalog/indexing.h"
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
@@ -32,6 +33,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
#include "utils/syscache.h"
#include "utils/tqual.h"
@@ -1420,3 +1422,44 @@ check_functional_grouping(Oid relid,
return false;
}
+
+static NameData *
+constraint_check_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ const char *constraint_name = (const char *) userdata;
+ NameData current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+ if (strncmp(NameStr(*version),
+ NameStr(current_version),
+ sizeof(NameData)) != 0)
+ ereport(WARNING,
+ (errmsg("constraint \"%s\" depends on collation %u version \"%s\", but the current version is \"%s\"",
+ constraint_name,
+ otherObject->objectId,
+ NameStr(*version),
+ NameStr(current_version)),
+ errdetail("The constraint may be corrupted due to changes in sort order."),
+ errhint("Drop and recreate the constraint to avoid the risk of corruption.")));
+
+ return NULL;
+}
+
+void
+constraint_check_collation_versions(Oid oid, const char *constraint_name)
+{
+ ObjectAddress object;
+
+ object.classId = ConstraintRelationId;
+ object.objectId = oid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &constraint_check_collation_version,
+ (void *) constraint_name);
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index b58d4f2af6d..c554704b771 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -4056,6 +4056,10 @@ CheckConstraintFetch(Relation relation)
check[found].ccbin = MemoryContextStrdup(CacheMemoryContext, s);
pfree(s);
+ /* Generate warnings if dependent collation versions have moved. */
+ constraint_check_collation_versions(HeapTupleGetOid(htup),
+ NameStr(conform->conname));
+
found++;
}
diff --git a/src/include/catalog/pg_constraint.h b/src/include/catalog/pg_constraint.h
index 66b3f13f74a..2126f503d35 100644
--- a/src/include/catalog/pg_constraint.h
+++ b/src/include/catalog/pg_constraint.h
@@ -261,4 +261,7 @@ extern bool check_functional_grouping(Oid relid,
List *grouping_columns,
List **constraintDeps);
+void
+constraint_check_collation_versions(Oid oid, const char *constraint_name);
+
#endif /* PG_CONSTRAINT_H */
--
2.17.0
0005-Use-glibc-version-in-lieu-of-collation-version-on-Li.patchapplication/octet-stream; name=0005-Use-glibc-version-in-lieu-of-collation-version-on-Li.patchDownload
From edfcaa8f9982ca54d5ff75a2a054827b16906051 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Thu, 6 Sep 2018 09:05:57 +1200
Subject: [PATCH 5/6] Use glibc version in lieu of collation version on Linux.
On systems using glibc (most GNU/Linux distributions), we can warn
users about possible index corruption due to collation changes by
using the glibc version string. Major operating system upgrades
and streaming replication between different operating system major
releases will result in warnings about the need to REINDEX, though
it's not a 100% reliable indicator, and there is no guarantee that
the collation definitions can't change without a change in the glibc
version (hopefully most distributions don't do that.)
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
configure | 2 +-
configure.in | 2 +-
src/backend/utils/adt/pg_locale.c | 13 +++++++++++++
src/include/pg_config.h.in | 3 +++
4 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/configure b/configure
index 9b304023d3d..7949f721b72 100755
--- a/configure
+++ b/configure
@@ -12704,7 +12704,7 @@ $as_echo "#define HAVE_STDBOOL_H 1" >>confdefs.h
fi
-for ac_header in atomic.h crypt.h fp_class.h getopt.h ieeefp.h ifaddrs.h langinfo.h mbarrier.h poll.h sys/epoll.h sys/ipc.h sys/prctl.h sys/procctl.h sys/pstat.h sys/resource.h sys/select.h sys/sem.h sys/shm.h sys/sockio.h sys/tas.h sys/un.h termios.h ucred.h utime.h wchar.h wctype.h
+for ac_header in atomic.h crypt.h fp_class.h getopt.h gnu/libc-version.h ieeefp.h ifaddrs.h langinfo.h mbarrier.h poll.h sys/epoll.h sys/ipc.h sys/prctl.h sys/procctl.h sys/pstat.h sys/resource.h sys/select.h sys/sem.h sys/shm.h sys/sockio.h sys/tas.h sys/un.h termios.h ucred.h utime.h wchar.h wctype.h
do :
as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default"
diff --git a/configure.in b/configure.in
index 2e60a89502c..d978438318c 100644
--- a/configure.in
+++ b/configure.in
@@ -1283,7 +1283,7 @@ AC_SUBST(UUID_LIBS)
AC_HEADER_STDBOOL
-AC_CHECK_HEADERS([atomic.h crypt.h fp_class.h getopt.h ieeefp.h ifaddrs.h langinfo.h mbarrier.h poll.h sys/epoll.h sys/ipc.h sys/prctl.h sys/procctl.h sys/pstat.h sys/resource.h sys/select.h sys/sem.h sys/shm.h sys/sockio.h sys/tas.h sys/un.h termios.h ucred.h utime.h wchar.h wctype.h])
+AC_CHECK_HEADERS([atomic.h crypt.h fp_class.h getopt.h gnu/libc-version.h ieeefp.h ifaddrs.h langinfo.h mbarrier.h poll.h sys/epoll.h sys/ipc.h sys/prctl.h sys/procctl.h sys/pstat.h sys/resource.h sys/select.h sys/sem.h sys/shm.h sys/sockio.h sys/tas.h sys/un.h termios.h ucred.h utime.h wchar.h wctype.h])
# On BSD, test for net/if.h will fail unless sys/socket.h
# is included first.
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index da651344e14..45faf088c1a 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -68,6 +68,10 @@
#include <unicode/ucnv.h>
#endif
+#ifdef HAVE_GNU_LIBC_VERSION_H
+#include <gnu/libc-version.h>
+#endif
+
#ifdef WIN32
/*
* This Windows file defines StrNCpy. We don't need it here, so we undefine
@@ -1438,6 +1442,15 @@ get_collation_actual_version(char collprovider, const char *collcollate)
}
else
#endif
+ if (collprovider == COLLPROVIDER_LIBC)
+ {
+#ifdef HAVE_GNU_LIBC_VERSION_H
+ collversion = gnu_get_libc_version();
+#else
+ collversion = NULL;
+#endif
+ }
+ else
collversion = NULL;
return collversion;
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 4094e22776c..9f05c24f7bc 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -290,6 +290,9 @@
/* Define to 1 if you have the `gettimeofday' function. */
#undef HAVE_GETTIMEOFDAY
+/* Define to 1 if you have the <gnu/libc-version.h> header file. */
+#undef HAVE_GNU_LIBC_VERSION_H
+
/* Define to 1 if you have the <gssapi/gssapi.h> header file. */
#undef HAVE_GSSAPI_GSSAPI_H
--
2.17.0
0006-Use-querylocale-to-get-collation-versions-on-FreeBSD.patchapplication/octet-stream; name=0006-Use-querylocale-to-get-collation-versions-on-FreeBSD.patchDownload
From 29745f26a097381b3bad32aaf773b33b4c5fc775 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Sun, 23 Sep 2018 21:55:21 +1200
Subject: [PATCH 6/6] Use querylocale() to get collation versions on FreeBSD.
Proposed feature for FreeBSD 13: https://reviews.freebsd.org/D17166
Details may change...
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
configure | 2 +-
configure.in | 2 +-
src/backend/utils/adt/pg_locale.c | 10 ++++++++++
src/include/pg_config.h.in | 3 +++
4 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/configure b/configure
index 7949f721b72..5071499f2a7 100755
--- a/configure
+++ b/configure
@@ -15093,7 +15093,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
+for ac_func in cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np querylocale readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
diff --git a/configure.in b/configure.in
index d978438318c..3a473d3cb0e 100644
--- a/configure.in
+++ b/configure.in
@@ -1562,7 +1562,7 @@ PGAC_FUNC_WCSTOMBS_L
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-AC_CHECK_FUNCS([cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
+AC_CHECK_FUNCS([cbrt clock_gettime fdatasync getifaddrs getpeerucred getrlimit mbstowcs_l memmove poll posix_fallocate pstat pthread_is_threaded_np querylocale readlink setproctitle setproctitle_fast setsid shm_open symlink sync_file_range utime utimes wcstombs_l])
AC_REPLACE_FUNCS(fseeko)
case $host_os in
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 45faf088c1a..38cd94495c9 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1446,6 +1446,16 @@ get_collation_actual_version(char collprovider, const char *collcollate)
{
#ifdef HAVE_GNU_LIBC_VERSION_H
collversion = gnu_get_libc_version();
+#elif defined(HAVE_QUERYLOCALE) && defined(LC_VERSION_MASK)
+ /* FreeBSD 13 */
+ locale_t loc = newlocale(LC_COLLATE, collcollate, NULL);
+ if (loc)
+ collversion =
+ pstrdup(querylocale(LC_COLLATE_MASK | LC_VERSION_MASK, loc));
+ else
+ ereport(ERROR,
+ (errmsg("could not load locale \"%s\"", collcollate)));
+ freelocale(loc);
#else
collversion = NULL;
#endif
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 9f05c24f7bc..9410793bd19 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -461,6 +461,9 @@
/* Have PTHREAD_PRIO_INHERIT. */
#undef HAVE_PTHREAD_PRIO_INHERIT
+/* Define to 1 if you have the `querylocale' function. */
+#undef HAVE_QUERYLOCALE
+
/* Define to 1 if you have the `random' function. */
#undef HAVE_RANDOM
--
2.17.0
Re: Collation versioning
On Tue, Sep 18, 2018 at 7:57 AM Douglas Doole <dougdoole@gmail.com> wrote:
On Mon, Sep 17, 2018 at 12:32 PM Greg Stark <stark@mit.edu> wrote:
This seems like a terrible idea in the open source world. Surely collation versioning means new ICU libraries can still provide the old collation rules so even if you update the library you can request the old version? We shouldn't need that actual old code with all its security holes and bugs just to get the old collation version.
We asked long and hard for this feature from the ICU team but they kept arguing it was too hard to do. There are apparently some tight couplings between the code and each version of CLDR. So the only way to support old collations is to ship the entire old library. (They even added make rules to allow the entire API to be version extended to accommodate this requirement.)
I wonder if this would be best modelled by entirely separate collation
entries with different OIDs, and possibly also separate collation
providers. Considering that to handle this we'd need to figure out
how link libicu.so.55, libicu.so.56, ... etc into the same backend,
and yet they presumably have the same collation names, doing it as
separate providers would create separate namespaces for their
collcollate values and reflect the reality that they really are
entirely independent providers. Admittedly that creates a whole can
of worms for initdb-time catalog creation, package maintainers' jobs,
how long old versions have to be supported and how you upgraded
database objects to new ICU versions. This kind of "major" versioning
with support for concurrently accessible major versions seems to be
different from the kind of version changes that happen under your feet
when libraries/collation definition files are updated.
Even bug fixes are potentially problematic because the fix may alter how some code points collate. The ICU team won't (or at least wouldn't - been a few years since I dealt with them) guarantee any sort of backwards compatibility between code drops.
Yeah, it seems like ICU is *also* subject to minor changes that happen
under your feet, much like libc. For example maintenance release 60.2
(you can't install that at the same time as 60.1, but you can install
it at the same time as 59.2). You'd be linked against libicu.so.60
(and thence libicudata.so.60), and it gets upgraded in place when you
run the local equivalent of apt-get upgrade.
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
It's been a bunch of years since I worked with ICU, so anything I say below
may have changed in their code or be subject to mental bit rot.
On Sun, Sep 23, 2018 at 2:48 PM Thomas Munro <thomas.munro@enterprisedb.com>
wrote:
Considering that to handle this we'd need to figure out
how link libicu.so.55, libicu.so.56, ... etc into the same backend,
and yet they presumably have the same collation names,
There's an option when compiling ICU to version extend the API names. So,
instead of calling ucol_open(), you'd call ucol_open_55() or ucol_open_56()
then you can link to both libixu.so.55 and libicu.so.56 without getting
name collisions.
The side effect of this is that it's the application's responsibility to
figure out which version of ICU "en_US" should be routed to. In DB2, we
started the collation names with UCAxxx (later changed to CLDRxxx) to let
us distinguish which version of the API to call.
Admittedly that creates a whole can
of worms for initdb-time catalog creation, package maintainers' jobs,
how long old versions have to be supported and how you upgraded
database objects to new ICU versions.
Yep. We never come up with a good answer for that before I left IBM. At the
time, DB2 only supported 2 or 3 version of ICU, so they were all shipped as
part of the install bundle.
Long term, I think the only viable approach to supporting multiple versions
of ICU is runtime loading of the libraries. Then it's up to the system
administrator to make sure the necessary versions are installed on the
system.
Yeah, it seems like ICU is *also* subject to minor changes that happen
under your feet, much like libc. For example maintenance release 60.2
(you can't install that at the same time as 60.1, but you can install
it at the same time as 59.2). You'd be linked against libicu.so.60
(and thence libicudata.so.60), and it gets upgraded in place when you
run the local equivalent of apt-get upgrade.
This always worried me because an unexpected collation change is so painful
for a database. And I was never able to think of a way of reliably testing
compatibility either because of ICU's ability to reorder and group
characters when collating.
Re: Collation versioning
On Tue, Sep 25, 2018 at 4:26 AM Douglas Doole <dougdoole@gmail.com> wrote:>
On Sun, Sep 23, 2018 at 2:48 PM Thomas Munro <thomas.munro@enterprisedb.com> wrote:
Admittedly that creates a whole can
of worms for initdb-time catalog creation, package maintainers' jobs,
how long old versions have to be supported and how you upgraded
database objects to new ICU versions.Yep. We never come up with a good answer for that before I left IBM. At the time, DB2 only supported 2 or 3 version of ICU, so they were all shipped as part of the install bundle.
Long term, I think the only viable approach to supporting multiple versions of ICU is runtime loading of the libraries. Then it's up to the system administrator to make sure the necessary versions are installed on the system.
I wonder if we would be practically constrained to using the
distro-supplied ICU (by their policies of not allowing packages to
ship their own copies ICU); it seems like it. I wonder which distros
allow multiple versions of ICU to be installed. I see that Debian 9.5
only has 57 in the default repo, but the major version is in the
package name (what is the proper term for that kind of versioning?)
and it doesn't declare a conflict with other versions, so that's
promising. Poking around with nm I noticed also that both the RHEL
and Debian ICU libraries have explicitly versioned symbol names like
"ucol_strcollUTF8_57", which is also promising. FreeBSD seems to have
used "--disable-renaming" and therefore defines only
"ucol_strcollUTF8"; doh.
This topic is discussed here:
http://userguide.icu-project.org/design#TOC-ICU-Binary-Compatibility:-Using-ICU-as-an-Operating-System-Level-Library
Personally I'm not planning to work on multi-version installation any
time soon, I was just scoping out some basic facts about all this. I
think the primary problem that affects most of our users is the
shifting-under-your-feet problem, which we now see applies equally to
libc and libicu.
Yeah, it seems like ICU is *also* subject to minor changes that happen
under your feet, much like libc. For example maintenance release 60.2
(you can't install that at the same time as 60.1, but you can install
it at the same time as 59.2). You'd be linked against libicu.so.60
(and thence libicudata.so.60), and it gets upgraded in place when you
run the local equivalent of apt-get upgrade.This always worried me because an unexpected collation change is so painful for a database. And I was never able to think of a way of reliably testing compatibility either because of ICU's ability to reorder and group characters when collating.
I think the best we can do is to track versions per dependency (ie
record it when the CHECK is created, when the index is created or
rebuilt, ...) and generate loud warnings until you've dealt with each
version dependency. That's why I've suggested we could consider
sticking it on pg_depend (though I have apparently failed to convince
Stephen so far). I think something like that is better than the
current collversion design, which punts the problem to the DBA: "hey,
human, there might be some problems, but I don't know where! Please
tell me when you've fixed them by running ALTER COLLATION ... REFRESH
VERSION!" instead of having the computer track of what actually needs
to be done on an object-by-object basis and update the versions
one-by-one automatically when the problems are resolved.
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
On Mon, Sep 24, 2018 at 1:47 PM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
Personally I'm not planning to work on multi-version installation any
time soon, I was just scoping out some basic facts about all this. I
think the primary problem that affects most of our users is the
shifting-under-your-feet problem, which we now see applies equally to
libc and libicu.
Are we sure about that? Could it just be that ICU will fix bugs that
cause their strcoll()-alike and strxfrm()-alike functions to give
behavior that isn't consistent with the behavior required by the CLDR
version in use?
This seems like it might be a very useful distinction. We know that
glibc had bugs that were caused by strxfrm() not agreeing with
strcoll() -- that was behind the 9.5-era abbreviated keys issues. But
that was actually a bug in an optimization in strcoll(), rather than a
strxfrm() bug. strxfrm() gave the correct answer, which is to say the
answer that was right according to the high level collation
definition. It merely failed to be bug-compatible with strcoll().
What's ICU supposed to do about an issue like that?
If we're going to continue to rely on the strxfrm() equivalent from
ICU, then it seems to me that ICU should be able to change behaviors
in a stable release, provided the behavior they're changing is down to
a bug in their infrastructure, as opposed to an organic evolution in
how some locale sorts text (CLDR update). My understanding is that ICU
is designed to decouple technical issues with issues of concern to
natural language experts, so we as an ICU client can limit ourselves
to worrying about one of the two at any given time.
--
Peter Geoghegan
Re: Collation versioning
Re: Thomas Munro 2018-09-24 <CAEepm=04PvEdmRmCCcn4c7ydDA=-G=uLe5vDdfJiqp58Jpi8Kw@mail.gmail.com>
I wonder if we would be practically constrained to using the
distro-supplied ICU (by their policies of not allowing packages to
ship their own copies ICU); it seems like it. I wonder which distros
allow multiple versions of ICU to be installed. I see that Debian 9.5
only has 57 in the default repo, but the major version is in the
package name (what is the proper term for that kind of versioning?)
and it doesn't declare a conflict with other versions, so that's
promising.
The point of the put-the-version/soname-into-the-package-name business
is to allow co-installation of (mostly library) packages[*], so this
is indeed possible with the ICU package on Debian.
The bad news is that this applies to the binary package name only, the
source package (from which the binaries are built) is only called
"icu". As Debian only ships one version of a source package in a
release, there can only be one libicu$version.deb in a release.
(This means you can have libicu52.deb and libicu60.deb installed in
parallel, but after upgrading, libicu52 won't have any installation
source. You can either keep or remove the package, but not reinstall
it.)
The fix would be to include the version in the source package name as
well, like postgresql-NN and llvm-toolchain-NN. (And then find a
maintainer willing to maintain the bunch.)
Christoph
[*] now historical footnote: this wasn't the case with the
libperl-5.xx packages which conflicted with each other, which is why
upgrading Debian used to remove the postgresql-plperl-$oldmajor
version on upgrade. This has now been fixed for the stretch->buster
upgrade.
Re: Collation versioning
On 16/09/2018 10:19, Thomas Munro wrote:
4. After creating a new database, update that row as appropriate in
the new database (!). Or find some other way to write a new table out
and switch it around, or something like that. That is, if you say
CREATE DATABASE foo LC_COLLATE = 'xx_XX', COLLATION_PROVIDER = libc
then those values somehow get written into the default pg_collation
row in the *new* database (so at that point it's not a simple copy of
the template database).
I've been hatching this exact scheme since the very beginning, even
thinking about using the background session functionality to do this.
It would solve a lot of problems, but there is the question of exactly
how to do that "(!)" part.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On 16/09/2018 20:12, Douglas Doole wrote:
All this collation stuff is great, and I know users want it, but it
feels like were pushing them out of an airplane with a ripped parachute
every time the collation libraries change. Maybe they'll land safely or
maybe things will get very messy.
At some point, a schema designer also needs to take some responsibility
for making smart choices for longevity. It is known that collations can
change, and the sort of changes that can happen are also generally
understood. So if you want to use range partitioning on text fields,
maybe you shouldn't, or at least choose the ranges conservatively.
Similarly, maybe you shouldn't have timestamp range partition boundaries
around DST changes or on the 29th of every month, and maybe you
shouldn't partition float values at negative zero. Some ideas are
better than others. We will help you recognize and fix breakage, but we
can't prevent it altogether.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On Fri, Sep 28, 2018 at 9:19 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 16/09/2018 10:19, Thomas Munro wrote:
4. After creating a new database, update that row as appropriate in
the new database (!). Or find some other way to write a new table out
and switch it around, or something like that. That is, if you say
CREATE DATABASE foo LC_COLLATE = 'xx_XX', COLLATION_PROVIDER = libc
then those values somehow get written into the default pg_collation
row in the *new* database (so at that point it's not a simple copy of
the template database).I've been hatching this exact scheme since the very beginning, even
thinking about using the background session functionality to do this.
It would solve a lot of problems, but there is the question of exactly
how to do that "(!)" part.
If that turns out to be impractical, I guess the "status quo" option
would be to add datcollprovider to pg_database. If we switch to
per-index version tracking as I proposed upthread (dropping
collversion), then the
where-do-we-stick-the-default-collation's-version problem goes away.
--
Thomas Munro
http://www.enterprisedb.com
Re: Collation versioning
Re: Thomas Munro 2018-09-27 <CAEepm=0EVCF_Nj5uYV5f6xH34MK1Z4mCfb+Svn1yJ_zsx5tOFw@mail.gmail.com>
4. After creating a new database, update that row as appropriate in
the new database (!). Or find some other way to write a new table out
and switch it around, or something like that.I've been hatching this exact scheme since the very beginning, even
thinking about using the background session functionality to do this.
It would solve a lot of problems, but there is the question of exactly
how to do that "(!)" part.
Making (!) work would also allow reassigning the "public" schema to
the database owner. That would fix that gross security gap that is
left with the default search_path, while still keeping usability. It
would make a whole lot of sense to work on making this feasible.
Christoph
Re: Collation versioning
On Fri, Sep 28, 2018 at 9:30 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 16/09/2018 20:12, Douglas Doole wrote:
All this collation stuff is great, and I know users want it, but it
feels like were pushing them out of an airplane with a ripped parachute
every time the collation libraries change. Maybe they'll land safely or
maybe things will get very messy.At some point, a schema designer also needs to take some responsibility
for making smart choices for longevity. It is known that collations can
change, and the sort of changes that can happen are also generally
understood. So if you want to use range partitioning on text fields,
maybe you shouldn't, or at least choose the ranges conservatively.
Similarly, maybe you shouldn't have timestamp range partition boundaries
around DST changes or on the 29th of every month, and maybe you
shouldn't partition float values at negative zero. Some ideas are
better than others. We will help you recognize and fix breakage, but we
can't prevent it altogether.
Since there's a chance of an "unconference" session on locale versions
tomorrow at PGCon, here's a fresh rebase of the patchset to add
per-database-object collation version tracking. It doesn't handle
default collations yet (not hard AFAIK, will try that soon), but it
does work well enough to demonstrate the generate principal. I won't
attach the CHECK support just yet, because it needs more work, but the
point of it was to demonstrate that pg_depend can handle this for all
kinds of database objects in one standard way, rather than sprinkling
collation version stuff all over the place in pg_index, pg_constraint,
etc, and I think it did that already.
postgres=# create table t (k text collate "en-x-icu");
CREATE TABLE
postgres=# create index on t(k);
CREATE INDEX
postgres=# select refobjversion from pg_depend where refobjversion != '';
refobjversion
---------------
153.72
(1 row)
Mess with it artificially (or install a different version of ICU):
postgres=# update pg_depend set refobjversion = '42' where
refobjversion = '153.72';
UPDATE 1
In a new session, we get a warning when first loading the index
because the version doesn't match:
postgres=# select * from t where k = 'x';
psql: WARNING: index "t_k_idx" depends on collation 12711 version
"42", but the current version is "153.72"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
k
---
(0 rows)
The warning can be cleared for the indexes on that one table like so:
postgres=# reindex table t;
REINDEX
You can see that it's captured the new version:
postgres=# select refobjversion from pg_depend where refobjversion != '';
refobjversion
---------------
153.72
(1 row)
--
Thomas Munro
https://enterprisedb.com
Attachments:
0001-Remove-pg_collation.collversion.patchapplication/octet-stream; name=0001-Remove-pg_collation.collversion.patchDownload
From ee07f7ba1770f881fd2025a9089df2b3d1a9029e Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/4] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ----
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 53 -----------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 ---------------------------
src/backend/nodes/copyfuncs.c | 13 ----
src/backend/nodes/equalfuncs.c | 11 ----
src/backend/parser/gram.y | 18 +-----
src/backend/tcop/utility.c | 12 ----
src/backend/utils/adt/pg_locale.c | 37 ------------
src/include/catalog/pg_collation.dat | 7 +--
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ----
15 files changed, 5 insertions(+), 270 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 4c7e93892ab..e41f414178a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2119,17 +2119,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index a79e7c0380b..4a8d6a8188d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21243,10 +21243,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index b51b3a25647..4241ec9f5a3 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,62 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index dd99d53547f..e67db0d2918 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 919e092483a..4f1442df035 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 78deade89b4..229f875d630 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3171,16 +3171,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5189,9 +5179,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 4f2ebe5118e..9bbcca1adff 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1099,14 +1099,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3249,9 +3241,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8311b1dd467..0376ee1b1df 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -825,7 +825,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10206,21 +10205,6 @@ DropdbStmt: DROP DATABASE database_name
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9578b5c7619..ded31d2a217 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1682,10 +1682,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = CreateStatistics((CreateStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2783,10 +2779,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3387,10 +3379,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2376bda497b..96439274ad2 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1357,8 +1357,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1460,41 +1458,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 367ce3607bc..7f88a41ad40 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index d3366f361d1..44c3b5fd85c 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index a9e633d6bb5..f17f7ef7ab4 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index e25f5d50b32..8090a5cb0c2 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2a8edf934f6..18e4a4cef00 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1850,17 +1850,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
--
2.21.0
0002-Add-pg_depend.refobjversion.patchapplication/octet-stream; name=0002-Add-pg_depend.refobjversion.patchDownload
From 30d43b1146d5e750a796d81dad7019f2a04edf58 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/4] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 20 +++++++++--
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 5 +++
src/include/catalog/pg_depend.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
6 files changed, 55 insertions(+), 30 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6315fc4b2fd..0333afe0b29 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1576,7 +1576,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1660,7 +1661,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies */
if (!ignore_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
free_object_addresses(self_addrs);
@@ -1668,7 +1670,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2611,7 +2614,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f7caedcc02c..d6af124446b 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -101,9 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ values[Anum_pg_depend_refobjversion - 1] = version ? NameGetDatum(version) : CStringGetDatum("");
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index ad5cd4194a7..4e9f707ade1 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1589,55 +1589,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ef9c86864cd..ef191a047b4 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,13 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced, const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index f786445fb29..e30896178a9 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -55,6 +55,7 @@ CATALOG(pg_depend,2608,DependRelationId)
Oid refclassid; /* OID of table containing object */
Oid refobjid; /* OID of object itself */
int32 refobjsubid; /* column number, or 0 if not used */
+ NameData refobjversion; /* version tracking, or empty if not used */
/*
* Precise semantics of the relationship are specified by the deptype
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8c..44d5e756a31 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | refobjversion | deptype
+---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
-- **************** pg_shdepend ****************
--
2.21.0
0003-Track-collation-versions-for-indexes.patchapplication/octet-stream; name=0003-Track-collation-versions-for-indexes.patchDownload
From ace00a51257cc802b25da98b4f4c73bf6088dc39 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 3/4] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. Whenever we load an index into the
relcache, check if the collation versions still match those reported
by the collation provider. Warn that the index may be corrupted if
not.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 57 ++++++++++++++++++++++
src/backend/catalog/index.c | 78 +++++++++++++++++++++++++++++-
src/backend/utils/adt/pg_locale.c | 21 ++++++++
src/backend/utils/cache/relcache.c | 5 ++
src/include/catalog/dependency.h | 8 +++
src/include/catalog/index.h | 2 +
src/include/utils/pg_locale.h | 1 +
7 files changed, 171 insertions(+), 1 deletion(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 0333afe0b29..3c947aa4e0c 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -437,6 +437,63 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = heap_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ NameData *new_version;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ new_version = callback(&otherObject, &foundDep->refobjversion,
+ userdata);
+ if (new_version)
+ {
+ /* Make a modifyable copy. */
+ tup = heap_copytuple(tup);
+ foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ foundDep->refobjversion = *new_version;
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ heap_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 50c8bb9ce65..a54a20d5876 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/syscache.h"
@@ -1079,11 +1080,15 @@ index_create(Relation heapRelation,
if (OidIsValid(collationObjectId[i]) &&
collationObjectId[i] != DEFAULT_COLLATION_OID)
{
+ NameData version;
+
referenced.classId = CollationRelationId;
referenced.objectId = collationObjectId[i];
referenced.objectSubId = 0;
+ get_collation_version_for_oid(referenced.objectId, &version);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnVersion(&myself, &referenced, &version,
+ DEPENDENCY_NORMAL);
}
}
@@ -1186,6 +1191,74 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static NameData *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ NameData current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+ if (strncmp(NameStr(*version),
+ NameStr(current_version),
+ sizeof(NameData)) != 0)
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation %u version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ otherObject->objectId,
+ NameStr(*version),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static NameData *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NameData *current_version = (NameData *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ get_collation_version_for_oid(otherObject->objectId, current_version);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3446,6 +3519,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 96439274ad2..1b846206508 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1510,6 +1510,27 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collatoin OID.
+ */
+void
+get_collation_version_for_oid(Oid oid, NameData *output)
+{
+ HeapTuple tp;
+ Form_pg_collation collform;
+ const char *version;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ memset(output, 0, sizeof(NameData));
+ if (version)
+ strncpy(NameStr(*output), version, sizeof(NameData));
+ ReleaseSysCache(tp);
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 2b992d78327..dc713977e91 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -1471,6 +1472,10 @@ RelationInitIndexAccessInfo(Relation relation)
indcoll = (oidvector *) DatumGetPointer(indcollDatum);
memcpy(relation->rd_indcollation, indcoll->values, indnkeyatts * sizeof(Oid));
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsCatalogRelation(relation))
+ index_check_collation_versions(RelationGetRelid(relation));
+
/*
* indclass cannot be referenced directly through the C struct, because it
* comes after the variable-width indkey field. Must extract the datum
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ef191a047b4..5d3f7411efe 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -176,6 +176,14 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef NameData *(*VisitDependentObjectsFun)(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 1113d25b2d8..4ae79aa5195 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -119,6 +119,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index b4b3aa5843e..023ef693e19 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern void get_collation_version_for_oid(Oid collid, NameData *output);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
--
2.21.0
Re: Collation versioning
On 2018-09-05 23:18, Thomas Munro wrote:
On Wed, Sep 5, 2018 at 12:10 PM Christoph Berg <myon@debian.org> wrote:
So, it's not ideal but perhaps worth considering on the grounds that
it's better than nothing?Ack.
Ok, here's a little patch like that.
postgres=# select collname, collcollate, collversion from pg_collation
where collname = 'en_NZ';
collname | collcollate | collversion
----------+-------------+-------------
en_NZ | en_NZ.utf8 | 2.24
(1 row)
After, um, briefly sleeping on this, I would like to go ahead with this.
There is ongoing work to make ICU available globally, and as part of
that I've also proposed a way to make the collation version tracking
work on a database level.
This here would be a useful piece on the overall picture. Independent
of what becomes of the ICU effort, we could have glibc collation version
tracking completely working for PG13.
The only open question on this patch was whether it's a good version to
use. I think based on subsequent discussions, there was the realization
that this is the best we can do and better than nothing.
In the patch, I would skip the configure test and just do
#ifdef __GLIBC__
directly.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On Thu, Oct 3, 2019 at 7:53 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 2018-09-05 23:18, Thomas Munro wrote:
On Wed, Sep 5, 2018 at 12:10 PM Christoph Berg <myon@debian.org> wrote:
So, it's not ideal but perhaps worth considering on the grounds that
it's better than nothing?Ack.
Ok, here's a little patch like that.
postgres=# select collname, collcollate, collversion from pg_collation
where collname = 'en_NZ';
collname | collcollate | collversion
----------+-------------+-------------
en_NZ | en_NZ.utf8 | 2.24
(1 row)After, um, briefly sleeping on this, I would like to go ahead with this.
There is ongoing work to make ICU available globally, and as part of
that I've also proposed a way to make the collation version tracking
work on a database level.This here would be a useful piece on the overall picture. Independent
of what becomes of the ICU effort, we could have glibc collation version
tracking completely working for PG13.
+1
Also, better ideas about which objects to attach versions to can come
along independently of this.
The only open question on this patch was whether it's a good version to
use. I think based on subsequent discussions, there was the realization
that this is the best we can do and better than nothing.In the patch, I would skip the configure test and just do
#ifdef __GLIBC__
directly.
Ok. Here's one like that. Also, a WIP patch for FreeBSD.
Thanks,
Thomas
Attachments:
0001-Use-libc-version-as-a-collation-version-on-glibc-sys.patchtext/x-patch; charset=US-ASCII; name=0001-Use-libc-version-as-a-collation-version-on-glibc-sys.patchDownload
From 79ec43cd69e82867f479e4df7e87789f89525826 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 4 Oct 2019 00:03:04 +1300
Subject: [PATCH 1/2] Use libc version as a collation version on glibc systems.
Using glibc's version number to detect potential collation definition
changes is not 100% reliable, but it's better than nothing.
Author: Thomas Munro
Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/4b76c6d4-ae5e-0dc6-7d0d-b5c796a07e34%402ndquadrant.com
---
src/backend/utils/adt/pg_locale.c | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index b2f08ead45..694ff7626e 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -70,6 +70,10 @@
#include <unicode/ucnv.h>
#endif
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+#endif
+
#ifdef WIN32
/*
* This Windows file defines StrNCpy. We don't need it here, so we undefine
@@ -1518,7 +1522,7 @@ pg_newlocale_from_collation(Oid collid)
char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
- char *collversion;
+ char *collversion = NULL;
#ifdef USE_ICU
if (collprovider == COLLPROVIDER_ICU)
@@ -1542,7 +1546,13 @@ get_collation_actual_version(char collprovider, const char *collcollate)
}
else
#endif
- collversion = NULL;
+ if (collprovider == COLLPROVIDER_LIBC)
+ {
+#if defined(__GLIBC__)
+ /* Use the glibc version because we don't have anything better. */
+ collversion = pstrdup(gnu_get_libc_version());
+#endif
+ }
return collversion;
}
--
2.20.1
0002-WIP-Use-querylocale-for-collation-versions-on-FreeBS.patchtext/x-patch; charset=US-ASCII; name=0002-WIP-Use-querylocale-for-collation-versions-on-FreeBS.patchDownload
From 81f5cf0283541874f6c9d2c2339e9326ff1ebee4 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 4 Oct 2019 00:19:49 +1300
Subject: [PATCH 2/2] WIP: Use querylocale() for collation versions on FreeBSD.
Requires a WIP FreeBSD patch: https://reviews.freebsd.org/D17166
---
src/backend/utils/adt/pg_locale.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 694ff7626e..a66f1c1fbe 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1551,6 +1551,18 @@ get_collation_actual_version(char collprovider, const char *collcollate)
#if defined(__GLIBC__)
/* Use the glibc version because we don't have anything better. */
collversion = pstrdup(gnu_get_libc_version());
+#elif defined(LC_VERSION_MASK)
+ /* FreeBSD exposes the CLDR version. */
+ locale_t loc = newlocale(LC_COLLATE, collcollate, NULL);
+ if (loc)
+ {
+ collversion =
+ pstrdup(querylocale(LC_COLLATE_MASK | LC_VERSION_MASK, loc));
+ freelocale(loc);
+ }
+ else
+ ereport(ERROR,
+ (errmsg("could not load locale \"%s\"", collcollate)));
#endif
}
--
2.20.1
Re: Collation versioning
On 2019-10-03 14:25, Thomas Munro wrote:
The only open question on this patch was whether it's a good version to
use. I think based on subsequent discussions, there was the realization
that this is the best we can do and better than nothing.In the patch, I would skip the configure test and just do
#ifdef __GLIBC__
directly.
Ok. Here's one like that.
Pushed that.
Also, a WIP patch for FreeBSD.
That looks promising.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On 2019-10-09 21:19, Peter Eisentraut wrote:
On 2019-10-03 14:25, Thomas Munro wrote:
The only open question on this patch was whether it's a good version to
use. I think based on subsequent discussions, there was the realization
that this is the best we can do and better than nothing.In the patch, I would skip the configure test and just do
#ifdef __GLIBC__
directly.
Ok. Here's one like that.
Pushed that.
Actually, I had to revert that because pg_dump and pg_upgrade tests need
to be updated, but that seems doable.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On Thu, Oct 10, 2019 at 8:38 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 2019-10-09 21:19, Peter Eisentraut wrote:
On 2019-10-03 14:25, Thomas Munro wrote:
The only open question on this patch was whether it's a good version to
use. I think based on subsequent discussions, there was the realization
that this is the best we can do and better than nothing.In the patch, I would skip the configure test and just do
#ifdef __GLIBC__
directly.
Ok. Here's one like that.
Pushed that.
Actually, I had to revert that because pg_dump and pg_upgrade tests need
to be updated, but that seems doable.
[Returning from a couple of weeks mostly away from computers]
Right, sorry about that. Here is a new version that fixes that test,
and also gives credit to Christoph for the idea in the commit message.
While testing pg_upgrade scenarios I noticed that initdb-created
collations' versions are not preserved, potentially losing track of
information about corrupted indexes. That's a preexisting condition,
and probably well understood, but it made me realise that if we switch
to per-database object (for example: per index) version tracking as
mentioned up-thread, then we should probably preserve that information
across pg_upgrade.
Attachments:
0001-Use-libc-version-as-a-collation-version-on-glibc--v2.patchapplication/octet-stream; name=0001-Use-libc-version-as-a-collation-version-on-glibc--v2.patchDownload
From bf932607f2a4a36a2c622ca6f09461227bd9d60e Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 4 Oct 2019 00:03:04 +1300
Subject: [PATCH] Use libc version as a collation version on glibc systems.
Using glibc's version number to detect potential collation definition
changes is not 100% reliable, but it's better than nothing.
Author: Thomas Munro, based on a suggestion from Christoph Berg
Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/4b76c6d4-ae5e-0dc6-7d0d-b5c796a07e34%402ndquadrant.com
---
src/backend/utils/adt/pg_locale.c | 14 ++++++++++++--
src/bin/pg_dump/t/002_pg_dump.pl | 4 ++--
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2a076a3dfd..fcdbaae37b 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -70,6 +70,10 @@
#include <unicode/ucnv.h>
#endif
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+#endif
+
#ifdef WIN32
/*
* This Windows file defines StrNCpy. We don't need it here, so we undefine
@@ -1499,7 +1503,7 @@ pg_newlocale_from_collation(Oid collid)
char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
- char *collversion;
+ char *collversion = NULL;
#ifdef USE_ICU
if (collprovider == COLLPROVIDER_ICU)
@@ -1523,7 +1527,13 @@ get_collation_actual_version(char collprovider, const char *collcollate)
}
else
#endif
- collversion = NULL;
+ if (collprovider == COLLPROVIDER_LIBC)
+ {
+#if defined(__GLIBC__)
+ /* Use the glibc version because we don't have anything better. */
+ collversion = pstrdup(gnu_get_libc_version());
+#endif
+ }
return collversion;
}
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 4712e3a958..1163420491 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -1376,8 +1376,8 @@ my %tests = (
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
- regexp => qr/^
- \QCREATE COLLATION public.test0 (provider = libc, locale = 'C');\E/xm,
+ regexp =>
+ qr/CREATE COLLATION public.test0 \(provider = libc, locale = 'C'(, version = '[^']*')?\);/m,
collation => 1,
like => { %full_runs, section_pre_data => 1, },
},
--
2.20.1
Re: Collation versioning
Re: Thomas Munro 2019-10-11 <CA+hUKGKDe98DFWKJoS7e4Z+Oamzc-1sZfpL3V3PPgi1uNvQ1tw@mail.gmail.com>
While testing pg_upgrade scenarios I noticed that initdb-created
collations' versions are not preserved, potentially losing track of
information about corrupted indexes. That's a preexisting condition,
and probably well understood, but it made me realise that if we switch
to per-database object (for example: per index) version tracking as
mentioned up-thread, then we should probably preserve that information
across pg_upgrade.
That would make much sense, yes. The whole problem is already complex
enough, if we add another "but if you use pg_upgrade, you still need
to do the tracking manually" footnote, users will be very confused.
Christoph
Re: Collation versioning
On Fri, Oct 11, 2019 at 11:41 PM Thomas Munro <thomas.munro@gmail.com> wrote:
On Thu, Oct 10, 2019 at 8:38 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:Actually, I had to revert that because pg_dump and pg_upgrade tests need
to be updated, but that seems doable.[Returning from a couple of weeks mostly away from computers]
Right, sorry about that. Here is a new version that fixes that test,
and also gives credit to Christoph for the idea in the commit message.
Here's a version with a small note added to the documentation. I'm
planning to commit this tomorrow.
To actually make this useful for most users, we need version tracking
for the default collation. I noticed that the ICU-as-default patch[1]/messages/by-id/attachment/104646/v1-0002-Add-option-to-use-ICU-as-global-collation-provide_rebased.patch
can do that for ICU collations (though I haven't looked closely yet).
Currently, its change to get_collation_actual_version() for the
default collation applies only when the default provider is ICU, but
if you just take out that condition when rebasing it should do the
right thing, I think?
Attachments:
0001-Use-libc-version-as-a-collation-version-on-glibc--v3.patchapplication/octet-stream; name=0001-Use-libc-version-as-a-collation-version-on-glibc--v3.patchDownload
From 4d7b1305b9850c9084ef2012bc54b18e70c59b0f Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Tue, 15 Oct 2019 16:31:48 +1300
Subject: [PATCH] Use libc version as a collation version on glibc systems.
Using glibc's version number to detect potential collation definition
changes is not 100% reliable, but it's better than nothing. More work
will be needed to handle the default collation.
Author: Thomas Munro, based on a suggestion from Christoph Berg
Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/4b76c6d4-ae5e-0dc6-7d0d-b5c796a07e34%402ndquadrant.com
---
doc/src/sgml/ref/alter_collation.sgml | 10 ++++++++++
src/backend/utils/adt/pg_locale.c | 14 ++++++++++++--
src/bin/pg_dump/t/002_pg_dump.pl | 4 ++--
3 files changed, 24 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index b51b3a2564..4cfcb42251 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -129,6 +129,16 @@ HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg
does not actually check whether all affected objects have been rebuilt
correctly.
</para>
+ <para>
+ When using collations provided by <literal>libc</literal> and
+ <productname>PostgreSQL</productname> was built with the GNU C library, the
+ C library's version is used as a collation version. Since collation
+ definitions typically change only with GNU C library releases, this provides
+ some defense against corruption, but it is not completely reliable.
+ </para>
+ <para>
+ Currently, there is no version tracking for the database default collation.
+ </para>
<para>
The following query can be used to identify all collations in the current
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2a076a3dfd..fcdbaae37b 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -70,6 +70,10 @@
#include <unicode/ucnv.h>
#endif
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+#endif
+
#ifdef WIN32
/*
* This Windows file defines StrNCpy. We don't need it here, so we undefine
@@ -1499,7 +1503,7 @@ pg_newlocale_from_collation(Oid collid)
char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
- char *collversion;
+ char *collversion = NULL;
#ifdef USE_ICU
if (collprovider == COLLPROVIDER_ICU)
@@ -1523,7 +1527,13 @@ get_collation_actual_version(char collprovider, const char *collcollate)
}
else
#endif
- collversion = NULL;
+ if (collprovider == COLLPROVIDER_LIBC)
+ {
+#if defined(__GLIBC__)
+ /* Use the glibc version because we don't have anything better. */
+ collversion = pstrdup(gnu_get_libc_version());
+#endif
+ }
return collversion;
}
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 4712e3a958..1163420491 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -1376,8 +1376,8 @@ my %tests = (
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
- regexp => qr/^
- \QCREATE COLLATION public.test0 (provider = libc, locale = 'C');\E/xm,
+ regexp =>
+ qr/CREATE COLLATION public.test0 \(provider = libc, locale = 'C'(, version = '[^']*')?\);/m,
collation => 1,
like => { %full_runs, section_pre_data => 1, },
},
--
2.20.1
Re: Collation versioning
On Tue, Oct 15, 2019 at 5:39 PM Thomas Munro <thomas.munro@gmail.com> wrote:
Here's a version with a small note added to the documentation. I'm
planning to commit this tomorrow.
Done.
It's not much, but it's a start. Some things to do:
* handle default collation (probably comes with CF entry 2256?)
* preserve versions of initdb-created collations in pg_upgrade
* ditch collversion and ALTER ... REFRESH VERSION and start tracking
versions dependencies per-index (etc)
Re: Collation versioning
On Wed, Oct 16, 2019 at 5:33 PM Thomas Munro <thomas.munro@gmail.com> wrote:
On Tue, Oct 15, 2019 at 5:39 PM Thomas Munro <thomas.munro@gmail.com> wrote:
Here's a version with a small note added to the documentation. I'm
planning to commit this tomorrow.Done.
The buildfarm is telling me that I didn't test this with the full set
of locales installed, so it fails on some systems. Will fix.
Re: Collation versioning
Hello Thomas,
On Tue, May 28, 2019 at 9:00 PM Thomas Munro <thomas.munro@gmail.com> wrote:
Since there's a chance of an "unconference" session on locale versions
tomorrow at PGCon, here's a fresh rebase of the patchset to add
per-database-object collation version tracking. It doesn't handle
default collations yet (not hard AFAIK, will try that soon), but it
does work well enough to demonstrate the generate principal. I won't
attach the CHECK support just yet, because it needs more work, but the
point of it was to demonstrate that pg_depend can handle this for all
kinds of database objects in one standard way, rather than sprinkling
collation version stuff all over the place in pg_index, pg_constraint,
etc, and I think it did that already.
Are you planning to continue working on it? For the record, that's
something needed to be able to implement a filter in REINDEX command
[1]: /messages/by-id/a81069b1-fdaa-ff40-436e-7840bd639ccf@2ndquadrant.com
I'm not sending a review since the code isn't finished yet, but one
issue with current approach is that the WARNING message recommending
to issue a REINDEX can be issued when running the required REINDEX,
which is at best unhelpful:
# update pg_depend set refobjversion = 'a' || refobjversion where
refobjversion != '';
# reindex table t1;
WARNING: 01000: index "t1_val_idx" depends on collation 13330 version
"a153.97.35.8", but the current version is "153.97.35.8"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
LOCATION: index_check_collation_version, index.c:1263
[1]: /messages/by-id/a81069b1-fdaa-ff40-436e-7840bd639ccf@2ndquadrant.com
Re: Collation versioning
On Fri, Nov 1, 2019 at 2:21 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
Are you planning to continue working on it? For the record, that's
something needed to be able to implement a filter in REINDEX command
[1].
Bonjour Julien,
Unfortunately I haven't had time to work on it seriously, but here's a
quick rebase to get the proof-of-concept back into working shape.
It's nice to see progress in other bits of the problem-space. I hope
to have time to look at this patch set again soon, but if you or
someone else would like hack on or think about it too, please feel
free!
Yes indeed this is exactly the same problem that you're trying to
solve, approached from a different starting point.
Here are some problems to think about:
* We'd need to track dependencies on the default collation once we
have versioning for that (see
/messages/by-id/5e756dd6-0e91-d778-96fd-b1bcb06c161a@2ndquadrant.com).
That is how most people actually consume collations out there in real
life, and yet we don't normally track dependencies on the default
collation and I don't know if that's simply a matter of ripping out
all the code that looks like "xxx != DEFAULT_COLLATION_ID" in the
dependency analysis code or if there's more to it.
* Andres mentioned off-list that pg_depend rows might get blown away
and recreated in some DDL circumstances. We need to look into that.
* Another is that pg_upgrade won't preserve pg_depend rows, so you'd
need some catalog manipulation (direct or via new DDL) to fix that.
* Some have expressed doubt that pg_depend is the right place for
this; let's see if any counter-proposals appear.
# reindex table t1;
WARNING: 01000: index "t1_val_idx" depends on collation 13330 version
"a153.97.35.8", but the current version is "153.97.35.8"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
LOCATION: index_check_collation_version, index.c:1263
Duh. Yeah, that's stupid and needs to be fixed somehow.
Attachments:
0001-Remove-pg_collation.collversion-v2.patchapplication/octet-stream; name=0001-Remove-pg_collation.collversion-v2.patchDownload
From 147c0077c8a3315688399ac1bc258338ad0dee0d Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/3] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ----
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 ---------------------------
src/backend/nodes/copyfuncs.c | 13 ----
src/backend/nodes/equalfuncs.c | 11 ----
src/backend/parser/gram.y | 18 +-----
src/backend/tcop/utility.c | 12 ----
src/backend/utils/adt/pg_locale.c | 37 ------------
src/include/catalog/pg_collation.dat | 7 +--
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ----
15 files changed, 5 insertions(+), 280 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 55694c4368..21adb1640a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2127,17 +2127,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 28eb322f3f..7e1c88bb82 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21660,10 +21660,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index dd99d53547..e67db0d291 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 919e092483..4f1442df03 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 3432bb921d..872a2d8dfc 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3175,16 +3175,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5163,9 +5153,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 18cb014373..ffb3d914c8 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1099,14 +1099,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3259,9 +3251,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 3f67aaf30e..27c8aae569 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -825,7 +825,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10235,21 +10234,6 @@ DropdbStmt: DROP DATABASE database_name
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index f2269ad35c..8bfed8da00 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1690,10 +1690,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2787,10 +2783,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3399,10 +3391,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index fcdbaae37b..b7a4a1421e 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 367ce3607b..7f88a41ad4 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index d3366f361d..44c3b5fd85 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index cc5dfed0bf..1fd27d5fd3 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index e25f5d50b3..8090a5cb0c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index d93a79a554..16a3b46bb3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1850,17 +1850,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
--
2.23.0
0002-Add-pg_depend.refobjversion-v2.patchapplication/octet-stream; name=0002-Add-pg_depend.refobjversion-v2.patchDownload
From 46932255252150df88a28206f5bb8c3de590f18d Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/3] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 20 +++++++++--
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 5 +++
src/include/catalog/pg_depend.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
6 files changed, 55 insertions(+), 30 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 03582781f6..516680bdfb 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2647,7 +2650,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index a060c25d2e..f1dc1143a7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -101,9 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ values[Anum_pg_depend_refobjversion - 1] = version ? NameGetDatum(version) : CStringGetDatum("");
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 88a261d9bd..0e13dedb9a 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1565,55 +1565,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ff50d594f6..a2ab071fef 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,13 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced, const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index f786445fb2..e30896178a 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -55,6 +55,7 @@ CATALOG(pg_depend,2608,DependRelationId)
Oid refclassid; /* OID of table containing object */
Oid refobjid; /* OID of object itself */
int32 refobjsubid; /* column number, or 0 if not used */
+ NameData refobjversion; /* version tracking, or empty if not used */
/*
* Precise semantics of the relationship are specified by the deptype
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..44d5e756a3 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | refobjversion | deptype
+---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
-- **************** pg_shdepend ****************
--
2.23.0
0003-Track-collation-versions-for-indexes-v2.patchapplication/octet-stream; name=0003-Track-collation-versions-for-indexes-v2.patchDownload
From 0946c482f51f91d4dabd97083116db77bbd56238 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 3/3] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. Whenever we load an index into the
relcache, check if the collation versions still match those reported
by the collation provider. Warn that the index may be corrupted if
not.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 57 ++++++++++++++++++++++
src/backend/catalog/index.c | 78 +++++++++++++++++++++++++++++-
src/backend/utils/adt/pg_locale.c | 21 ++++++++
src/backend/utils/cache/relcache.c | 5 ++
src/include/catalog/dependency.h | 8 +++
src/include/catalog/index.h | 2 +
src/include/utils/pg_locale.h | 1 +
7 files changed, 171 insertions(+), 1 deletion(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 516680bdfb..9572a4d8b5 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -437,6 +437,63 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ NameData *new_version;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ new_version = callback(&otherObject, &foundDep->refobjversion,
+ userdata);
+ if (new_version)
+ {
+ /* Make a modifyable copy. */
+ tup = heap_copytuple(tup);
+ foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ foundDep->refobjversion = *new_version;
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7c34509696..cefb18a3dd 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/syscache.h"
@@ -1124,11 +1125,15 @@ index_create(Relation heapRelation,
if (OidIsValid(collationObjectId[i]) &&
collationObjectId[i] != DEFAULT_COLLATION_OID)
{
+ NameData version;
+
referenced.classId = CollationRelationId;
referenced.objectId = collationObjectId[i];
referenced.objectSubId = 0;
+ get_collation_version_for_oid(referenced.objectId, &version);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnVersion(&myself, &referenced, &version,
+ DEPENDENCY_NORMAL);
}
}
@@ -1231,6 +1236,74 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static NameData *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ NameData current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+ if (strncmp(NameStr(*version),
+ NameStr(current_version),
+ sizeof(NameData)) != 0)
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation %u version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ otherObject->objectId,
+ NameStr(*version),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static NameData *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NameData *current_version = (NameData *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ get_collation_version_for_oid(otherObject->objectId, current_version);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3548,6 +3621,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index b7a4a1421e..f7aa3a3672 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1501,6 +1501,27 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collatoin OID.
+ */
+void
+get_collation_version_for_oid(Oid oid, NameData *output)
+{
+ HeapTuple tp;
+ Form_pg_collation collform;
+ const char *version;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ memset(output, 0, sizeof(NameData));
+ if (version)
+ strncpy(NameStr(*output), version, sizeof(NameData));
+ ReleaseSysCache(tp);
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 585dcee5db..5d4a728ffc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -1470,6 +1471,10 @@ RelationInitIndexAccessInfo(Relation relation)
indcoll = (oidvector *) DatumGetPointer(indcollDatum);
memcpy(relation->rd_indcollation, indcoll->values, indnkeyatts * sizeof(Oid));
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsCatalogRelation(relation))
+ index_check_collation_versions(RelationGetRelid(relation));
+
/*
* indclass cannot be referenced directly through the C struct, because it
* comes after the variable-width indkey field. Must extract the datum
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a2ab071fef..7d7009442b 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -176,6 +176,14 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef NameData *(*VisitDependentObjectsFun)(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 1113d25b2d..4ae79aa519 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -119,6 +119,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index b4b3aa5843..023ef693e1 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern void get_collation_version_for_oid(Oid collid, NameData *output);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
--
2.23.0
Re: Collation versioning
Hello Thomas,
On Mon, Nov 4, 2019 at 4:58 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Fri, Nov 1, 2019 at 2:21 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
Are you planning to continue working on it? For the record, that's
something needed to be able to implement a filter in REINDEX command
[1].Bonjour Julien,
Unfortunately I haven't had time to work on it seriously, but here's a
quick rebase to get the proof-of-concept back into working shape.
It's nice to see progress in other bits of the problem-space. I hope
to have time to look at this patch set again soon, but if you or
someone else would like hack on or think about it too, please feel
free!
Thanks! I already did some hack on it when looking at the code so I
can try to make some progress.
Yes indeed this is exactly the same problem that you're trying to
solve, approached from a different starting point.Here are some problems to think about:
* We'd need to track dependencies on the default collation once we
have versioning for that (see
/messages/by-id/5e756dd6-0e91-d778-96fd-b1bcb06c161a@2ndquadrant.com).
That is how most people actually consume collations out there in real
life, and yet we don't normally track dependencies on the default
collation and I don't know if that's simply a matter of ripping out
all the code that looks like "xxx != DEFAULT_COLLATION_ID" in the
dependency analysis code or if there's more to it.
This isn't enough. What would remain is:
- teach get_collation_version_for_oid() to return the default
collation name, which is simple
- have recordDependencyOnVersion() actually records the dependency,
which wouldn't happen as the default collation is pinned.
An easy fix would be to teach isObjectPinned() to ignore
CollationRelationId / DEFAULT_COLLATION_OID, but that's ugly and would
allow too many dependencies to be stored. Not pinning the default
collation during initdb doesn't sound a good alternative either.
Maybe adding a force flag or a new DependencyType that'd mean "normal
but forced" would be ok?
* Andres mentioned off-list that pg_depend rows might get blown away
and recreated in some DDL circumstances. We need to look into that.
* Another is that pg_upgrade won't preserve pg_depend rows, so you'd
need some catalog manipulation (direct or via new DDL) to fix that.
* Some have expressed doubt that pg_depend is the right place for
this; let's see if any counter-proposals appear.
When working on the REINDEX FILTER, I totally missed this thread and
wrote a POC saving the version in pg_index. That's not ideal though,
as you need to record multiple version strings. In my version I used
a json type, using the collprovider as the key, but that's not enough
for ICU as each collation can have a different version string. I'm
not a huge fan of using pg_depend to record the version, but storing a
collprovider/collname -> version per row in pg_index is definitely a
no go, so I don't have any better counter-proposal.
# reindex table t1;
WARNING: 01000: index "t1_val_idx" depends on collation 13330 version
"a153.97.35.8", but the current version is "153.97.35.8"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
LOCATION: index_check_collation_version, index.c:1263Duh. Yeah, that's stupid and needs to be fixed somehow.
I don't have a clever solution for that either.
Re: Collation versioning
On Mon, Nov 4, 2019 at 11:13 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Mon, Nov 4, 2019 at 4:58 AM Thomas Munro <thomas.munro@gmail.com> wrote:
Here are some problems to think about:
* We'd need to track dependencies on the default collation once we
have versioning for that [...]
Another problem I just thought about is how to avoid discrepancy of
collation version for indexes on shared objects, such as
pg_database_datname_index.
Re: Collation versioning
On Tue, Nov 5, 2019 at 4:18 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Mon, Nov 4, 2019 at 11:13 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Mon, Nov 4, 2019 at 4:58 AM Thomas Munro <thomas.munro@gmail.com> wrote:
Here are some problems to think about:
* We'd need to track dependencies on the default collation once we
have versioning for that [...]Another problem I just thought about is how to avoid discrepancy of
collation version for indexes on shared objects, such as
pg_database_datname_index.
I didn't look closely at the code, but I think when "name" recently
became collation-aware (commit 586b98fd), it switched to using
C_COLLATION_OID as its typcollation, and "C" doesn't need versioning,
so I think it would only be a problem if there are shared catalogs
that have "name" columns that have a non-type-default collation.
There are none of those, and you can't create them, right? If there
were, if we take this patch set to its logical conclusion, we'd also
need pg_shdepend.refobjversion, but we don't need it AFAICS.
Re: Collation versioning
On Mon, Nov 4, 2019 at 11:13 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Mon, Nov 4, 2019 at 4:58 AM Thomas Munro <thomas.munro@gmail.com> wrote:
* Some have expressed doubt that pg_depend is the right place for
this; let's see if any counter-proposals appear.When working on the REINDEX FILTER, I totally missed this thread and
wrote a POC saving the version in pg_index. That's not ideal though,
as you need to record multiple version strings. In my version I used
a json type, using the collprovider as the key, but that's not enough
for ICU as each collation can have a different version string. I'm
not a huge fan of using pg_depend to record the version, but storing a
collprovider/collname -> version per row in pg_index is definitely a
no go, so I don't have any better counter-proposal.
Yeah, I also thought about using pg_index directly, and was annoyed by
the denormalisation you mention (an array of {collation, version}!?)
and so I realised I wanted another table like they teach you at
database school, but I also realised that there are other kinds of
database objects that depend on collations and that can become
corrupted if the collation definition changes. It was thinking about
that that lead me to the idea of using something that can record
version dependences on *any* database object, which brought me to the
existing pg_depend table.
Concretely, eventually we might want to support checks etc, as
mentioned by Doug Doole and as I showed in an earlier version of this
POC patch, though I removed it from the more recent patch set so we
can focus on the more pressing problems. The check constraint idea
leads to more questions like: "does this constraint *really* use any
operators that truly depend on the collation definition?" (so CHECK
(name > 'xxx') depends on name's collation, but CHECK (LENGTH(name) <
32) doesn't really), and I didn't want to be distracted by that rabbit
hole. Here's the example message that came out of the earlier patch
for posterity:
WARNING: constraint "t_i_check" depends on collation 12018 version
"30.0.1", but the current version is "30.0.2"
DETAIL: The constraint may be corrupted due to changes in sort order.
HINT: Drop and recreate the constraint to avoid the risk of corruption.
Re: Collation versioning
On Mon, Nov 4, 2019 at 11:13 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
When working on the REINDEX FILTER, I totally missed this thread and
I created a new wiki page to try to track the various moving pieces
here. Julien, Peter, Christoph, anyone interested, please feel free
to update it or add more information.
Re: Collation versioning
On 2019-11-04 04:58, Thomas Munro wrote:
* We'd need to track dependencies on the default collation once we
have versioning for that (see
/messages/by-id/5e756dd6-0e91-d778-96fd-b1bcb06c161a@2ndquadrant.com).
That is how most people actually consume collations out there in real
life, and yet we don't normally track dependencies on the default
collation and I don't know if that's simply a matter of ripping out
all the code that looks like "xxx != DEFAULT_COLLATION_ID" in the
dependency analysis code or if there's more to it.
As I was working on that lately, I came to the conclusion that we should
get *this* patch done first.
My patch for default collation versioning had the version of the default
collation in the pg_collation record for the "default" collation. But
that way you can't set the collation version during CREATE DATABASE.
It's also pretty complicated (but not impossible) to get the collation
version in template1 set during initdb. So you'd need a new mechanism,
perhaps to store it in pg_database instead.
So instead of going through all those complications of creating this new
mechanism, only to rip it out again not much later, we should focus on
moving the per-object tracking forward. That would solve these problems
because you don't need to track the version at database creation time,
only when you create objects using the collations.
* Some have expressed doubt that pg_depend is the right place for
this; let's see if any counter-proposals appear.
The only alternative is to create a new catalog that contains exactly
the same columns as pg_depend (minus deptype) plus the version. That
would work but it would just create a lot of code duplication, I think.
One thing I've been thinking about is whether this object-version
concept could extend to other object types. For example, if someone
changes the binary layout of a type, they could change the version of
the type, and this catalog could track the type version in the column ->
type dependency. Obviously, a lot more work would have to be done to
make this work, but I think the concept of this catalog is sound.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On Mon, Nov 4, 2019 at 9:59 PM Thomas Munro <thomas.munro@gmail.com> wrote:
On Tue, Nov 5, 2019 at 4:18 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Mon, Nov 4, 2019 at 11:13 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Mon, Nov 4, 2019 at 4:58 AM Thomas Munro <thomas.munro@gmail.com> wrote:
Here are some problems to think about:
* We'd need to track dependencies on the default collation once we
have versioning for that [...]Another problem I just thought about is how to avoid discrepancy of
collation version for indexes on shared objects, such as
pg_database_datname_index.I didn't look closely at the code, but I think when "name" recently
became collation-aware (commit 586b98fd), it switched to using
C_COLLATION_OID as its typcollation, and "C" doesn't need versioning,
so I think it would only be a problem if there are shared catalogs
that have "name" columns that have a non-type-default collation.
There are none of those, and you can't create them, right? If there
were, if we take this patch set to its logical conclusion, we'd also
need pg_shdepend.refobjversion, but we don't need it AFAICS.
That's entirely correct, I should have checked that.
Re: Collation versioning
On Mon, Nov 4, 2019 at 11:13 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Mon, Nov 4, 2019 at 4:58 AM Thomas Munro <thomas.munro@gmail.com> wrote:
Unfortunately I haven't had time to work on it seriously, but here's a
quick rebase to get the proof-of-concept back into working shape.
Thanks! I also removed the test for REFRESH VERSION command that was
forgotten in the patch set, and run a pgindent.
Here are some problems to think about:
* We'd need to track dependencies on the default collation once we
have versioning for that (see
/messages/by-id/5e756dd6-0e91-d778-96fd-b1bcb06c161a@2ndquadrant.com).
That is how most people actually consume collations out there in real
life, and yet we don't normally track dependencies on the default
collation and I don't know if that's simply a matter of ripping out
all the code that looks like "xxx != DEFAULT_COLLATION_ID" in the
dependency analysis code or if there's more to it.This isn't enough. What would remain is:
- teach get_collation_version_for_oid() to return the default
collation name, which is simple
- have recordDependencyOnVersion() actually records the dependency,
which wouldn't happen as the default collation is pinned.An easy fix would be to teach isObjectPinned() to ignore
CollationRelationId / DEFAULT_COLLATION_OID, but that's ugly and would
allow too many dependencies to be stored. Not pinning the default
collation during initdb doesn't sound a good alternative either.
Maybe adding a force flag or a new DependencyType that'd mean "normal
but forced" would be ok?
Attached 4th patch handles default collation. I went with an
ignore_systempin flag in recordMultipleDependencies.
* Andres mentioned off-list that pg_depend rows might get blown away
and recreated in some DDL circumstances. We need to look into that.
I tried various flavour of DDL but I couldn't wipe out the pg_depend
rows without having an index rebuild triggered (like changing the
underlying column datatype). Do you have any scenario where the index
rebuild wouldn't be triggered?
* Another is that pg_upgrade won't preserve pg_depend rows, so you'd
need some catalog manipulation (direct or via new DDL) to fix that.
Attached 5th patch add a new "ALTER INDEX idx_name DEPENDS ON
COLLATION coll_oid VERSION coll_version_text" that can only be
executed in binary upgrade mode, and teach pg_dump to generate such
commands (including for indexes created for constraints). One issue
is that older versions don't have pg_depend information, so pg_dump
can't find the dependencies to generate such commands and override the
version with anything else. It means that the upgraded cluster will
show all indexes as depending on the current collation provider
version. I'm not sure if that's the best thing to do, or if we should
change all pg_depend rows to mention "unknown" version or something
like that. It would generate so much noise that it's probably not
worth it.
I didn't do anything for the spurious warning when running a reindex,
and kept original approach of pg_depend catalog.
Attachments:
0002-Add-pg_depend.refobjversion.patchapplication/octet-stream; name=0002-Add-pg_depend.refobjversion.patchDownload
From f1fc16b777a2af8c696c4e6df691cf07005c3cea Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 20 +++++++++--
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
6 files changed, 56 insertions(+), 30 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 03582781f6..516680bdfb 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2647,7 +2650,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index a060c25d2e..f1dc1143a7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -101,9 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ values[Anum_pg_depend_refobjversion - 1] = version ? NameGetDatum(version) : CStringGetDatum("");
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 88a261d9bd..0e13dedb9a 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1565,55 +1565,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ff50d594f6..b5d3276e77 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index f786445fb2..e30896178a 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -55,6 +55,7 @@ CATALOG(pg_depend,2608,DependRelationId)
Oid refclassid; /* OID of table containing object */
Oid refobjid; /* OID of object itself */
int32 refobjsubid; /* column number, or 0 if not used */
+ NameData refobjversion; /* version tracking, or empty if not used */
/*
* Precise semantics of the relationship are specified by the deptype
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..44d5e756a3 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | refobjversion | deptype
+---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
0005-Preserve-index-dependencies-on-collation-during-pg_u.patchapplication/octet-stream; name=0005-Preserve-index-dependencies-on-collation-during-pg_u.patchDownload
From 650c05c8c2c9d214860c6db6d8ab7709c42ad9e3 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/5] Preserve index dependencies on collation during
pg_upgrade
A new command ALTER INDEX i_name DEPENDS ON COLLATION c_oid VERSION v_name is
added to force a dependency on a collation specific version, which is only
allowed in binary upgrade mode.
Also teach pg_dump to emit such commands for all indexes, including indexes
created for constraints, when run with --binary-upgrade flag.
---
src/backend/commands/tablecmds.c | 64 +++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 9 ++
src/bin/pg_dump/pg_dump.c | 153 ++++++++++++++++++++++++++++---
src/bin/pg_dump/pg_dump.h | 3 +
src/include/nodes/parsenodes.h | 4 +-
6 files changed, 222 insertions(+), 12 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 5597be6e3d..1e94c0f5e2 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -203,6 +203,13 @@ typedef struct NewColumnValue
ExprState *exprstate; /* execution state */
} NewColumnValue;
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ NameData version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/*
* Error-reporting support for RemoveRelations
*/
@@ -531,6 +538,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecDependsOnCollationVersion(Relation rel, Oid oid, char *version);
/* ----------------------------------------------------------------
@@ -3799,6 +3807,11 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ /* Only used in binary upgrade mode */
+ case AT_DependsOnCollationVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -3952,6 +3965,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
+ * VERSION ... */
+ if (!IsBinaryUpgrade)
+ ereport(ERROR,
+ (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ (errmsg("command can only be called when server is in binary upgrade mode"))));
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode);
@@ -4476,6 +4499,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_DependsOnCollationVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecDependsOnCollationVersion(rel, cmd->oid, cmd->name);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -16723,3 +16751,39 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+static NameData *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* We only care about dependencies on a specific collation. */
+ if (otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return &forced_dependency->version;
+}
+
+static void
+ATExecDependsOnCollationVersion(Relation rel, Oid oid, char *version)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ forced_dependency.oid = oid;
+ strncpy(NameStr(forced_dependency.version), version, sizeof(NameData));
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 872a2d8dfc..fd3b535eff 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3167,6 +3167,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
COPY_SCALAR_FIELD(num);
+ COPY_SCALAR_FIELD(oid);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
COPY_SCALAR_FIELD(behavior);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 27c8aae569..7299499868 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2547,6 +2547,15 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> DEPENDS ON COLLATION ... VERSION ... */
+ | DEPENDS ON COLLATION Iconst VERSION_P IDENT
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->name = $6;
+ n->oid = (Oid)$4;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index bf69adc2f4..8ad5319885 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -288,6 +288,7 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -6841,7 +6842,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6877,7 +6880,7 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6902,7 +6905,64 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = 1259 AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = 3456 AND "
+ " refobjversion != '') AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(refobjversion ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = 1259 AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = 3456 AND "
+ " refobjversion != '') AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "'' AS inddependoids, "
+ "'' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6941,7 +7001,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'' AS inddependoids, "
+ "'' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6976,7 +7038,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'' AS inddependoids, "
+ "'' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7007,7 +7071,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'' AS inddependoids, "
+ "'' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7041,7 +7107,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'' AS inddependoids, "
+ "'' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7081,6 +7149,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7106,6 +7176,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16372,10 +16444,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16435,6 +16508,9 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16464,6 +16540,20 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18421,6 +18511,47 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of ALTER INDEX ... DEPENDS ON COLLATION ... VERSION ....
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ if (strcmp(inddependoids, "") == 0)
+ {
+ Assert(strcmp(inddependversions, "") == 0);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ appendPQExpBuffer(buffer, "ALTER INDEX %s ",
+ fmtQualifiedDumpable(indxinfo));
+ appendPQExpBuffer(buffer, " DEPENDS ON COLLATION %s VERSION %s\n",
+ inddependoidsarray[i],
+ fmtQualifiedId(NULL, inddependversionsarray[i]));
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 7b2c1524a5..6f67d195dd 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 16a3b46bb3..467ce01359 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1824,7 +1824,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_DependsOnCollationVersion /* DEPENDS ON COLLATION ... VERSION ... */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1842,6 +1843,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
* or tablespace */
int16 num; /* attribute number for columns referenced by
* number */
+ Oid oid; /* oid reference for collation dependency */
RoleSpec *newowner;
Node *def; /* definition of new column, index,
* constraint, or parent table */
--
2.20.1
0001-Remove-pg_collation.collversion.patchapplication/octet-stream; name=0001-Remove-pg_collation.collversion.patchDownload
From 9c09cfcbc779882d78d101d7f7e76383990891c0 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/5] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
17 files changed, 5 insertions(+), 288 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 55694c4368..21adb1640a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2127,17 +2127,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 28eb322f3f..7e1c88bb82 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21660,10 +21660,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index dd99d53547..e67db0d291 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 919e092483..4f1442df03 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 3432bb921d..872a2d8dfc 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3175,16 +3175,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5163,9 +5153,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 18cb014373..ffb3d914c8 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1099,14 +1099,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3259,9 +3251,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 3f67aaf30e..27c8aae569 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -825,7 +825,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10235,21 +10234,6 @@ DropdbStmt: DROP DATABASE database_name
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index e984545780..90acba5d8e 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1690,10 +1690,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2794,10 +2790,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3406,10 +3398,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index fcdbaae37b..b7a4a1421e 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 367ce3607b..7f88a41ad4 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index d3366f361d..44c3b5fd85 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index cc5dfed0bf..1fd27d5fd3 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index e25f5d50b3..8090a5cb0c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index d93a79a554..16a3b46bb3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1850,17 +1850,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
0004-Also-track-default-collation-version.patchapplication/octet-stream; name=0004-Also-track-default-collation-version.patchDownload
From d14d5408008e675c10d0b85cad8d5e7be1c4b114 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Fri, 1 Nov 2019 14:05:44 +0100
Subject: [PATCH 4/5] Also track default collation version.
---
src/backend/catalog/dependency.c | 12 +++++---
src/backend/catalog/index.c | 23 ++++++++++++----
src/backend/catalog/pg_depend.c | 13 +++++----
src/backend/utils/adt/pg_locale.c | 32 +++++++++++++++++-----
src/include/catalog/dependency.h | 6 ++--
src/test/regress/expected/create_index.out | 8 ++++--
6 files changed, 68 insertions(+), 26 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6287a4017a..518b53094f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1661,7 +1661,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs, NULL,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1749,7 +1750,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs, NULL,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ false);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1770,7 +1772,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs, NULL,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -2708,7 +2711,8 @@ record_object_address_dependencies(const ObjectAddress *depender,
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
referenced->refs, NULL, referenced->numrefs,
- behavior);
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index ee841a9a36..2e9b28fe11 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1118,14 +1118,25 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /*
+ * Store dependency on collations The C and posix collations are
+ * pinned, so don't bother recording them
+ */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collationObjectId[i])
+ && collationObjectId[i] != C_COLLATION_OID
+ && collationObjectId[i] != POSIX_COLLATION_OID
+ )
{
- NameData version;
+ NameData version;
+ bool force;
+
+ /*
+ * We need to force the recording for the default collation as
+ * it's pinned
+ */
+ force = (collationObjectId[i] == DEFAULT_COLLATION_OID);
referenced.classId = CollationRelationId;
referenced.objectId = collationObjectId[i];
@@ -1133,7 +1144,7 @@ index_create(Relation heapRelation,
get_collation_version_for_oid(referenced.objectId, &version);
recordDependencyOnVersion(&myself, &referenced, &version,
- DEPENDENCY_NORMAL);
+ DEPENDENCY_NORMAL, force);
}
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f1dc1143a7..db83658cf6 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior, false);
}
/*
@@ -57,9 +57,11 @@ void
recordDependencyOnVersion(const ObjectAddress *depender,
const ObjectAddress *referenced,
const NameData *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool ignore_systempin)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ recordMultipleDependencies(depender, referenced, version, 1, behavior,
+ ignore_systempin);
}
/*
@@ -71,7 +73,8 @@ recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool ignore_systempin)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -104,7 +107,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
* need to record dependencies on it. This saves lots of space in
* pg_depend, so it's worth the time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index f7aa3a3672..19a1a6573a 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -1508,15 +1510,31 @@ void
get_collation_version_for_oid(Oid oid, NameData *output)
{
HeapTuple tp;
- Form_pg_collation collform;
const char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
- if (!HeapTupleIsValid(tp))
- elog(ERROR, "cache lookup failed for collation %u", oid);
- collform = (Form_pg_collation) GETSTRUCT(tp);
- version = get_collation_actual_version(collform->collprovider,
- NameStr(collform->collcollate));
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
memset(output, 0, sizeof(NameData));
if (version)
strncpy(NameStr(*output), version, sizeof(NameData));
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 6ba9e634a5..83ea8c9458 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -193,13 +193,15 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordDependencyOnVersion(const ObjectAddress *depender,
const ObjectAddress *referenced,
const NameData *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool ignore_systempin);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool ignore_systempin);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 1cdb7a9663..2f30d66935 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -1990,15 +1990,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2018,15 +2020,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
--
2.20.1
0003-Track-collation-versions-for-indexes.patchapplication/octet-stream; name=0003-Track-collation-versions-for-indexes.patchDownload
From 3976e2c7e73ed457700abb707f39bc5183856356 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. Whenever we load an index into the
relcache, check if the collation versions still match those reported
by the collation provider. Warn that the index may be corrupted if
not.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 57 ++++++++++++++++++++++
src/backend/catalog/index.c | 78 +++++++++++++++++++++++++++++-
src/backend/utils/adt/pg_locale.c | 21 ++++++++
src/backend/utils/cache/relcache.c | 5 ++
src/include/catalog/dependency.h | 8 +++
src/include/catalog/index.h | 2 +
src/include/utils/pg_locale.h | 1 +
7 files changed, 171 insertions(+), 1 deletion(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 516680bdfb..6287a4017a 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -437,6 +437,63 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ NameData *new_version;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ new_version = callback(&otherObject, &foundDep->refobjversion,
+ userdata);
+ if (new_version)
+ {
+ /* Make a modifyable copy. */
+ tup = heap_copytuple(tup);
+ foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ foundDep->refobjversion = *new_version;
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7c34509696..ee841a9a36 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/syscache.h"
@@ -1124,11 +1125,15 @@ index_create(Relation heapRelation,
if (OidIsValid(collationObjectId[i]) &&
collationObjectId[i] != DEFAULT_COLLATION_OID)
{
+ NameData version;
+
referenced.classId = CollationRelationId;
referenced.objectId = collationObjectId[i];
referenced.objectSubId = 0;
+ get_collation_version_for_oid(referenced.objectId, &version);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnVersion(&myself, &referenced, &version,
+ DEPENDENCY_NORMAL);
}
}
@@ -1231,6 +1236,74 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static NameData *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ NameData current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+ if (strncmp(NameStr(*version),
+ NameStr(current_version),
+ sizeof(NameData)) != 0)
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation %u version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ otherObject->objectId,
+ NameStr(*version),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static NameData *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NameData *current_version = (NameData *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ get_collation_version_for_oid(otherObject->objectId, current_version);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3548,6 +3621,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index b7a4a1421e..f7aa3a3672 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1501,6 +1501,27 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collatoin OID.
+ */
+void
+get_collation_version_for_oid(Oid oid, NameData *output)
+{
+ HeapTuple tp;
+ Form_pg_collation collform;
+ const char *version;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ memset(output, 0, sizeof(NameData));
+ if (version)
+ strncpy(NameStr(*output), version, sizeof(NameData));
+ ReleaseSysCache(tp);
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 585dcee5db..5d4a728ffc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -1470,6 +1471,10 @@ RelationInitIndexAccessInfo(Relation relation)
indcoll = (oidvector *) DatumGetPointer(indcollDatum);
memcpy(relation->rd_indcollation, indcoll->values, indnkeyatts * sizeof(Oid));
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsCatalogRelation(relation))
+ index_check_collation_versions(RelationGetRelid(relation));
+
/*
* indclass cannot be referenced directly through the C struct, because it
* comes after the variable-width indkey field. Must extract the datum
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b5d3276e77..6ba9e634a5 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -176,6 +176,14 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef NameData *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 1113d25b2d..4ae79aa519 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -119,6 +119,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index b4b3aa5843..023ef693e1 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern void get_collation_version_for_oid(Oid collid, NameData *output);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
--
2.20.1
Re: Collation versioning
On Thu, Nov 7, 2019 at 10:20 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
Attached 4th patch handles default collation. I went with an
ignore_systempin flag in recordMultipleDependencies.
Thanks for working on this! I haven't looked closely or tested yet,
but this seems reasonable. Obviously it assumes that the default
provider is really "libc" in disguise for now, but Peter's other patch
will extend that to cover ICU later.
* Andres mentioned off-list that pg_depend rows might get blown away
and recreated in some DDL circumstances. We need to look into that.I tried various flavour of DDL but I couldn't wipe out the pg_depend
rows without having an index rebuild triggered (like changing the
underlying column datatype). Do you have any scenario where the index
rebuild wouldn't be triggered?
Ah, OK, if we only do that when the old index contents will also be
destroyed, that's great news.
* Another is that pg_upgrade won't preserve pg_depend rows, so you'd
need some catalog manipulation (direct or via new DDL) to fix that.Attached 5th patch add a new "ALTER INDEX idx_name DEPENDS ON
COLLATION coll_oid VERSION coll_version_text" that can only be
executed in binary upgrade mode, and teach pg_dump to generate such
commands (including for indexes created for constraints).
It's nice that you were able to make up a reasonable grammar out of
existing keywords. I wonder if we should make this user accessible...
it could be useful for expert users. If so, maybe it should use
collation names, not OIDs?
One issue
is that older versions don't have pg_depend information, so pg_dump
can't find the dependencies to generate such commands and override the
version with anything else. It means that the upgraded cluster will
show all indexes as depending on the current collation provider
version. I'm not sure if that's the best thing to do, or if we should
change all pg_depend rows to mention "unknown" version or something
like that. It would generate so much noise that it's probably not
worth it.
Right, so this is basically a policy decision: do we assume that all
pre-13 indexes that depend on collations are potentially corrupted, or
assume that they are not? The "correct" thing to do would be to
assume they are potentially corrupted and complain until the user
reindexes, but I think the pragmatic thing to do would be to assume
that they're not and just let them adopt the current versions, even
though it's a lie. I lean towards the pragmatic choice; we're trying
to catch future problems, not give the entire user base a load of
extra work to do on their next pg_upgrade for mostly theoretical
reasons. (That said, given the new glibc versioning, we'll
effectively be giving most of our user base a load of extra work to do
on their next OS upgrade and that'll be a characteristic of PostgreSQL
going forward, once the versioning-for-default-provider patch goes
in.) Any other opinions?
I didn't do anything for the spurious warning when running a reindex,
and kept original approach of pg_depend catalog.
I see three options:
1. Change all or some of index_open(), relation_open(),
RelationIdGetRelation(), RelationBuildDesc() and
RelationInitIndexAccessInfo() to take some kind of flag so we can say
NO_COLLATION_VERSION_CHECK_PLEASE, and then have ReindexIndex() pass
that flag down when opening it for the purpose of rebuilding it.
2. Use a global state to suppress these warnings while opening that
index. Perhaps ReindexIndex() would call RelCacheWarnings(false)
before index_open(), and use PG_FINALLY to make sure it restores
warnings with RelCacheWarnings(true). (This is a less-code-churn
bad-programming-style version of #1.)
3. Move the place that we run the collation version check. Instead
of doing it in RelationInitIndexAccessInfo() so that it runs when the
relation cache first loads the index, put it into a new routine
RelationCheckValidity() and call that from ... I don't know, some
other place that runs whenever we access indexes but not when we
rebuild them.
3's probably a better idea, if you can find a reasonable place to call
it from. I'm thinking some time around the time the executor acquires
locks, but using a flag in the relcache entry to make sure it doesn't
run the check again if there were no warnings last time (so one
successful version check turns the extra work off for the rest of this
relcache entry's lifetime). I think it'd be a feature, not a bug, if
it gave you the warning every single time you executed a query using
an index that has a mismatch... but a practical alternative would be
to check only once per index and that probably makes more sense.
Re: Collation versioning
On Thu, Nov 7, 2019 at 1:27 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
As I was working on that lately, I came to the conclusion that we should
get *this* patch done first.
Cool. Let's aim to get this into 13!
* Some have expressed doubt that pg_depend is the right place for
this; let's see if any counter-proposals appear.The only alternative is to create a new catalog that contains exactly
the same columns as pg_depend (minus deptype) plus the version. That
would work but it would just create a lot of code duplication, I think.
Agreed.
One thing I've been thinking about is whether this object-version
concept could extend to other object types. For example, if someone
changes the binary layout of a type, they could change the version of
the type, and this catalog could track the type version in the column ->
type dependency. Obviously, a lot more work would have to be done to
make this work, but I think the concept of this catalog is sound.
Interesting idea. Sounds like it requires version checks that
actually stop you from using the dependent object, instead of emitting
a few meek warnings.
Re: Collation versioning
On Fri, Nov 08, 2019 at 02:23:54PM +1300, Thomas Munro wrote:
Right, so this is basically a policy decision: do we assume that all
pre-13 indexes that depend on collations are potentially corrupted, or
assume that they are not? The "correct" thing to do would be to
assume they are potentially corrupted and complain until the user
reindexes, but I think the pragmatic thing to do would be to assume
that they're not and just let them adopt the current versions, even
though it's a lie. I lean towards the pragmatic choice; we're trying
to catch future problems, not give the entire user base a load of
extra work to do on their next pg_upgrade for mostly theoretical
reasons. (That said, given the new glibc versioning, we'll
effectively be giving most of our user base a load of extra work to do
on their next OS upgrade and that'll be a characteristic of PostgreSQL
going forward, once the versioning-for-default-provider patch goes
in.) Any other opinions?
Matching an incorrect collation version on an index which physically
uses something else does not strike me as a good idea to me because
you may hide corruptions, and you would actually lose the reason why
the corruption happened (did the version bump up from an incorrect
one? Or what?). Could it be possible to mark any existing indexes
with an unknown version or something like that? This way, we could
just let the user decide what needs to be reindexed or not, and we
need to offer an option to update the collation version from unknown
to the latest one available.
--
Michael
Re: Collation versioning
On Fri, Nov 8, 2019 at 2:37 PM Michael Paquier <michael@paquier.xyz> wrote:
On Fri, Nov 08, 2019 at 02:23:54PM +1300, Thomas Munro wrote:
Right, so this is basically a policy decision: do we assume that all
pre-13 indexes that depend on collations are potentially corrupted, or
assume that they are not? The "correct" thing to do would be to
assume they are potentially corrupted and complain until the user
reindexes, but I think the pragmatic thing to do would be to assume
that they're not and just let them adopt the current versions, even
though it's a lie. I lean towards the pragmatic choice; we're trying
to catch future problems, not give the entire user base a load of
extra work to do on their next pg_upgrade for mostly theoretical
reasons. (That said, given the new glibc versioning, we'll
effectively be giving most of our user base a load of extra work to do
on their next OS upgrade and that'll be a characteristic of PostgreSQL
going forward, once the versioning-for-default-provider patch goes
in.) Any other opinions?Matching an incorrect collation version on an index which physically
uses something else does not strike me as a good idea to me because
you may hide corruptions, and you would actually lose the reason why
the corruption happened (did the version bump up from an incorrect
one? Or what?). Could it be possible to mark any existing indexes
with an unknown version or something like that? This way, we could
just let the user decide what needs to be reindexed or not, and we
need to offer an option to update the collation version from unknown
to the latest one available.
Fair point.
So we have three proposals:
1. Assume that pre-13 indexes that depend on collations are
potentially corrupted and complain until they are reindexed. This
could be done by having pg_upgrade run ALTER INDEX ... DEPENDS ON
COLLATION "fr_FR" VERSION '' (empty string, or some other default
value that we don't think is going to coincide with a real version).
2. Assume that pre-13 indexes are not corrupted. In the target 13
database, the index will be created in the catalogs with the
provider's current version.
3. We don't know if pre-13 indexes are corrupted or not, and we'll
record that with a special value just as in proposal #1, except that
we could show a different hint for that special version value. It
would tell you can you can either REINDEX, or run ALTER INDEX ...
DEPENDS ON COLLATION "fr_FR" VERSION '34.0' if you believe the index
to have been created with the current collation version on an older
release of PostgreSQL that didn't track versions.
Re: Collation versioning
On Fri, 2019-11-08 at 15:04 +1300, Thomas Munro wrote:
So we have three proposals:
1. Assume that pre-13 indexes that depend on collations are
potentially corrupted and complain until they are reindexed. This
could be done by having pg_upgrade run ALTER INDEX ... DEPENDS ON
COLLATION "fr_FR" VERSION '' (empty string, or some other default
value that we don't think is going to coincide with a real version).
2. Assume that pre-13 indexes are not corrupted. In the target 13
database, the index will be created in the catalogs with the
provider's current version.
3. We don't know if pre-13 indexes are corrupted or not, and we'll
record that with a special value just as in proposal #1, except that
we could show a different hint for that special version value. It
would tell you can you can either REINDEX, or run ALTER INDEX ...
DEPENDS ON COLLATION "fr_FR" VERSION '34.0' if you believe the index
to have been created with the current collation version on an older
release of PostgreSQL that didn't track versions.
I think #1 is bad because it would frighten all users, even those who
didn't upgrade their libc at all, only PostgreSQL. I don't think that
shouting "wolf" for no real reason will improve trust in PostgreSQL.
#2 is bad because it may hide pre-existing index corruption.
#3 is the best proposal, but there is still the need to run
ALTER INDEX on all affected indexes to keep PostgreSQL from nagging.
Perhaps the situation could be improved with a pg_upgrade option
--i-know-my-indexes-are-fine that causes a result like #2.
Together with a bold note in the release notes, this may relieve
the pain.
Yours,
Laurenz Albe
Re: Collation versioning
Re: Laurenz Albe 2019-11-08 <3c3b9ff84d21acf3188558928249d04db84ea2e9.camel@cybertec.at>
#3 is the best proposal, but there is still the need to run
ALTER INDEX on all affected indexes to keep PostgreSQL from nagging.
Perhaps the situation could be improved with a pg_upgrade option
--i-know-my-indexes-are-fine that causes a result like #2.
Together with a bold note in the release notes, this may relieve
the pain.
Ack.
We should also try to make the actual commands more accessible.
Instead of having the user specify a version number we could as well
determine from the current state of the system as in
ALTER INDEX ... DEPENDS ON 'version-number-I-never-heard-of-before'
could it just be
ALTER INDEX ... COLLATION IS CURRENT
or, given the general action to take is reindexing, how about a no-op reindex?
REINDEX INDEX ... METADATA ONLY
That might look less scary to the average end user.
Do we even think people upgrade PG and the OS at the same time?
pg_upgrade might frequently actually be invoked on an otherwise
unchanged system, so we could even make "collations are fine" the
default for pg_upgrade. And maybe have a switch like pg_upgrade --os-upgrade
that reverses this.
Christoph
Re: Collation versioning
On Fri, Nov 8, 2019 at 2:24 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Thu, Nov 7, 2019 at 10:20 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
Attached 4th patch handles default collation. I went with an
ignore_systempin flag in recordMultipleDependencies.Thanks for working on this! I haven't looked closely or tested yet,
but this seems reasonable. Obviously it assumes that the default
provider is really "libc" in disguise for now, but Peter's other patch
will extend that to cover ICU later.
Yes, that should require minimal changes here.
* Andres mentioned off-list that pg_depend rows might get blown away
and recreated in some DDL circumstances. We need to look into that.I tried various flavour of DDL but I couldn't wipe out the pg_depend
rows without having an index rebuild triggered (like changing the
underlying column datatype). Do you have any scenario where the index
rebuild wouldn't be triggered?Ah, OK, if we only do that when the old index contents will also be
destroyed, that's great news.* Another is that pg_upgrade won't preserve pg_depend rows, so you'd
need some catalog manipulation (direct or via new DDL) to fix that.Attached 5th patch add a new "ALTER INDEX idx_name DEPENDS ON
COLLATION coll_oid VERSION coll_version_text" that can only be
executed in binary upgrade mode, and teach pg_dump to generate such
commands (including for indexes created for constraints).It's nice that you were able to make up a reasonable grammar out of
existing keywords. I wonder if we should make this user accessible...
it could be useful for expert users. If so, maybe it should use
collation names, not OIDs?
I thought about it, but I'm wondering if it's a good idea to expose
this to users. The command is not really POLA compliant, as what it
actually mean is "update the recorded version for all existing
pg_depend rows, if any, for this index and collation", while one could
assume that it'll add a new pg_depend row if none exist. Ideally,
users would only need to use command that says "trust me this index
(or collation version) is actually compatible with this collation's
current version" or similar, but not some user provided string.
I didn't do anything for the spurious warning when running a reindex,
and kept original approach of pg_depend catalog.I see three options:
1. Change all or some of index_open(), relation_open(),
RelationIdGetRelation(), RelationBuildDesc() and
RelationInitIndexAccessInfo() to take some kind of flag so we can say
NO_COLLATION_VERSION_CHECK_PLEASE, and then have ReindexIndex() pass
that flag down when opening it for the purpose of rebuilding it.
2. Use a global state to suppress these warnings while opening that
index. Perhaps ReindexIndex() would call RelCacheWarnings(false)
before index_open(), and use PG_FINALLY to make sure it restores
warnings with RelCacheWarnings(true). (This is a less-code-churn
bad-programming-style version of #1.)
3. Move the place that we run the collation version check. Instead
of doing it in RelationInitIndexAccessInfo() so that it runs when the
relation cache first loads the index, put it into a new routine
RelationCheckValidity() and call that from ... I don't know, some
other place that runs whenever we access indexes but not when we
rebuild them.3's probably a better idea, if you can find a reasonable place to call
it from. I'm thinking some time around the time the executor acquires
locks, but using a flag in the relcache entry to make sure it doesn't
run the check again if there were no warnings last time (so one
successful version check turns the extra work off for the rest of this
relcache entry's lifetime). I think it'd be a feature, not a bug, if
it gave you the warning every single time you executed a query using
an index that has a mismatch... but a practical alternative would be
to check only once per index and that probably makes more sense.
OTOH, 2 is more generic, and could maybe be a better way with Peter's
idea of new catalog that would also fit other use cases?
Re: Collation versioning
On Fri, Nov 8, 2019 at 10:20 AM Christoph Berg <myon@debian.org> wrote:
Re: Laurenz Albe 2019-11-08 <3c3b9ff84d21acf3188558928249d04db84ea2e9.camel@cybertec.at>
#3 is the best proposal, but there is still the need to run
ALTER INDEX on all affected indexes to keep PostgreSQL from nagging.
Perhaps the situation could be improved with a pg_upgrade option
--i-know-my-indexes-are-fine that causes a result like #2.
Together with a bold note in the release notes, this may relieve
the pain.Ack.
+1
We should also try to make the actual commands more accessible.
Instead of having the user specify a version number we could as well
determine from the current state of the system as in
ALTER INDEX ... DEPENDS ON 'version-number-I-never-heard-of-before'
This one is needed for pg_upgrade on later version, but I agree that
it shouldn't be exposed to users.
could it just be
ALTER INDEX ... COLLATION IS CURRENT
this sounds like a better idea, though this should probably work at
the collation lever rather than index level. I think that we should
offer users this but with multiple filter, like:
- mark all indexes' collation version dependencies as current version
- mark all indexes' dependencies on a specific collation and collation
version as current version
- mark all indexes' dependencies on a specific collation (any version)
as current version
or, given the general action to take is reindexing, how about a no-op reindex?
REINDEX INDEX ... METADATA ONLYThat might look less scary to the average end user.
This should be scary, as any misuse can lead to hidden corruption. If
a user isn't sure of what to do, a plain REINDEX is the safe (and
painful) way to go.
Do we even think people upgrade PG and the OS at the same time?
pg_upgrade might frequently actually be invoked on an otherwise
unchanged system, so we could even make "collations are fine" the
default for pg_upgrade. And maybe have a switch like pg_upgrade --os-upgrade
that reverses this.
+1
Re: Collation versioning
Some more thoughts:
1. If you create an index on an expression that includes a COLLATE or
a partial index that has one in the WHERE clause, you get bogus
warnings:
postgres=# create table t (v text);
CREATE TABLE
postgres=# create index on t(v) where v > 'hello' collate "en_NZ";
WARNING: index "t_v_idx3" depends on collation "en_NZ" version "",
but the current version is "2.28"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
CREATE INDEX
postgres=# create index on t((case when v < 'x' collate "en_NZ" then
'foo' else 'bar' end));
WARNING: index "t_case_idx" depends on collation "en_NZ" version "",
but the current version is "2.28"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
CREATE INDEX
That's because the 0003 patch only calls recordDependencyOnVersion()
for simple attribute references. When
recordDependencyOnSingleRelExpr() is called by index_create() to
analyse ii_Expressions and ii_Predicate, it's going to have to be
smart enough to detect collation references and record the versions.
There is also some more code that ignores pinned collations hiding in
there.
This leads to the difficult question of how you recognise a real
dependency on a collation's version in an expression. I have some
vague ideas but haven't seriously looked into it yet. (The same
question comes up for check constraint -> collation dependencies.)
2. If you create a composite type with a text attribute (with or
without an explicit collation), and then create an index on a column
of that type, we don't record the dependency.
postgres=# create type my_type as (x text collate "en_NZ");
CREATE TYPE
postgres=# create table t (v my_type);
CREATE TABLE
postgres=# create index on t(v);
CREATE INDEX
postgres=# select * from pg_depend where refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
I think create_index() will need to perform recursive analysis on
composite types to look for text attributes, when they appear as
simple attributes, and then add direct dependencies index -> collation
to capture the versions. Then we might need to do the same for
composite types hiding inside ii_Expressions and ii_Predicate (once we
figure out what that really means).
3. Test t/002_pg_dump.pl in src/bin/pg_upgrade fails.
4. In the warning message we should show get_collation_name() instead
of the OID.
Re: Collation versioning
On Sun, Nov 10, 2019 at 10:08 AM Thomas Munro <thomas.munro@gmail.com> wrote:
Some more thoughts:
1. If you create an index on an expression that includes a COLLATE or
a partial index that has one in the WHERE clause, you get bogus
warnings:postgres=# create table t (v text);
CREATE TABLE
postgres=# create index on t(v) where v > 'hello' collate "en_NZ";
WARNING: index "t_v_idx3" depends on collation "en_NZ" version "",
but the current version is "2.28"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
CREATE INDEXpostgres=# create index on t((case when v < 'x' collate "en_NZ" then
'foo' else 'bar' end));
WARNING: index "t_case_idx" depends on collation "en_NZ" version "",
but the current version is "2.28"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
CREATE INDEXThat's because the 0003 patch only calls recordDependencyOnVersion()
for simple attribute references. When
recordDependencyOnSingleRelExpr() is called by index_create() to
analyse ii_Expressions and ii_Predicate, it's going to have to be
smart enough to detect collation references and record the versions.
There is also some more code that ignores pinned collations hiding in
there.This leads to the difficult question of how you recognise a real
dependency on a collation's version in an expression. I have some
vague ideas but haven't seriously looked into it yet. (The same
question comes up for check constraint -> collation dependencies.)
Oh good point. A simple and exhaustive way to deal with that would be
to teach recordMultipleDependencies() to override isObjectPinned() and
retrieve the collation version if the referenced object is a collation
and it's neither C or POSIX collation. It means that we could also
remove the extra "version" argument and get rid of
recordDependencyOnVersion to simply call recordMultipleDependencies in
create_index for direct column references having a collation.
Would it be ok to add this kind of knowledge in
recordMultipleDependencies() or is it too hacky?
2. If you create a composite type with a text attribute (with or
without an explicit collation), and then create an index on a column
of that type, we don't record the dependency.postgres=# create type my_type as (x text collate "en_NZ");
CREATE TYPE
postgres=# create table t (v my_type);
CREATE TABLE
postgres=# create index on t(v);
CREATE INDEX
postgres=# select * from pg_depend where refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)I think create_index() will need to perform recursive analysis on
composite types to look for text attributes, when they appear as
simple attributes, and then add direct dependencies index -> collation
to capture the versions. Then we might need to do the same for
composite types hiding inside ii_Expressions and ii_Predicate (once we
figure out what that really means).
Isn't that actually a bug? For instance such an index will have a 0
indcollation in pg_index, and according to pg_index documentation:
" this contains the OID of the collation to use for the index, or zero
if the column is not of a collatable data type."
You can't use a COLLATE expression on such data type, but it still has
a collation used.
3. Test t/002_pg_dump.pl in src/bin/pg_upgrade fails.
Apparently neither "make check" nor "make world" run this test :(
This was broken due collversion support in pg_dump, I have fixed it
locally.
4. In the warning message we should show get_collation_name() instead
of the OID.
+1, I also fixed it locally.
Re: Collation versioning
On Wed, Nov 13, 2019 at 3:27 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Sun, Nov 10, 2019 at 10:08 AM Thomas Munro <thomas.munro@gmail.com> wrote:
That's because the 0003 patch only calls recordDependencyOnVersion()
for simple attribute references. When
recordDependencyOnSingleRelExpr() is called by index_create() to
analyse ii_Expressions and ii_Predicate, it's going to have to be
smart enough to detect collation references and record the versions.
There is also some more code that ignores pinned collations hiding in
there.This leads to the difficult question of how you recognise a real
dependency on a collation's version in an expression. I have some
vague ideas but haven't seriously looked into it yet. (The same
question comes up for check constraint -> collation dependencies.)Oh good point. A simple and exhaustive way to deal with that would be
to teach recordMultipleDependencies() to override isObjectPinned() and
retrieve the collation version if the referenced object is a collation
and it's neither C or POSIX collation. It means that we could also
remove the extra "version" argument and get rid of
recordDependencyOnVersion to simply call recordMultipleDependencies in
create_index for direct column references having a collation.Would it be ok to add this kind of knowledge in
recordMultipleDependencies() or is it too hacky?
That doesn't seem like the right place; that's a raw data insertion
function, though... I guess it does already have enough brains to skip
pinned objects. Hmm.
I think create_index() will need to perform recursive analysis on
composite types to look for text attributes, when they appear as
simple attributes, and then add direct dependencies index -> collation
to capture the versions. Then we might need to do the same for
composite types hiding inside ii_Expressions and ii_Predicate (once we
figure out what that really means).Isn't that actually a bug? For instance such an index will have a 0
indcollation in pg_index, and according to pg_index documentation:" this contains the OID of the collation to use for the index, or zero
if the column is not of a collatable data type."You can't use a COLLATE expression on such data type, but it still has
a collation used.
I don't think it's a bug. The information is available, but you have
to follow the graph to get it. Considering that the composite type
could be something like CREATE TYPE my_type AS (fr_name text COLLATE
"fr_CA", en_name text COLLATE "en_CA"), there is no single collation
you could put into pg_index.indcollation anyway. As for pg_depend,
it's currently enough for the index to depend on the type, and the
type to depend on the collation, because the purpose of dependencies
is to control dropping and dumping order, but for our new purpose we
also need to create direct dependencies index -> "fr_CA", index ->
"en_CA" so we can record the current versions.
3. Test t/002_pg_dump.pl in src/bin/pg_upgrade fails.
Apparently neither "make check" nor "make world" run this test :(
This was broken due collversion support in pg_dump, I have fixed it
locally.
make check-world
Re: Collation versioning
On Tue, Nov 12, 2019 at 10:16 PM Thomas Munro <thomas.munro@gmail.com> wrote:
On Wed, Nov 13, 2019 at 3:27 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Sun, Nov 10, 2019 at 10:08 AM Thomas Munro <thomas.munro@gmail.com> wrote:
That's because the 0003 patch only calls recordDependencyOnVersion()
for simple attribute references. When
recordDependencyOnSingleRelExpr() is called by index_create() to
analyse ii_Expressions and ii_Predicate, it's going to have to be
smart enough to detect collation references and record the versions.
There is also some more code that ignores pinned collations hiding in
there.
[...]
Indeed. Now, using a composite type in an expression index, I can see that eg.
CREATE TYPE mytype AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
CREATE TABLE test1(id integer, myval mytype);
CREATE INDEX ON sometable (somecol) WHERE (mytype).fr_name = 'meh'
does create a dependency on fr-x-icu collation, because collations are
checked for FieldSelect nodes (which indeed ignores default
collation), but eg.
CREATE INDEX idx2 ON test1(id) WHERE myval = ('foo', 'bar');
won't, so I confirm that recordDependencyOnSingleRelExpr() isn't
bullet proof either for finding collation dependencies.
I think create_index() will need to perform recursive analysis on
composite types to look for text attributes, when they appear as
simple attributes, and then add direct dependencies index -> collation
to capture the versions. Then we might need to do the same for
composite types hiding inside ii_Expressions and ii_Predicate (once we
figure out what that really means).
I did write some code that can extract all collations that are used by
a datatype, which seems to work as intended over many combinations of
composite / array / domain types used in index simple attributes. I'm
not sure if I should change find_expr_references_walker (called by
recordDependencyOnExpr) to also track those new dependencies in
ii_Expression and ii_Predicate, as it'll also add unneeded
dependencies for other callers. And if I should add version detection
there too or have recordMultipleDependencies() automatically take care
of this.
A simple and exhaustive way to deal with that would be
to teach recordMultipleDependencies() to override isObjectPinned() and
retrieve the collation version if the referenced object is a collation
and it's neither C or POSIX collationThat doesn't seem like the right place; that's a raw data insertion
function, though... I guess it does already have enough brains to skip
pinned objects. Hmm.
Another point is that unless we also do an additional check to see
what relkind is referencing the collation, it'll record a version
string for types and other objects.
Isn't that actually a bug? For instance such an index will have a 0
indcollation in pg_index, and according to pg_index documentation:" this contains the OID of the collation to use for the index, or zero
if the column is not of a collatable data type."You can't use a COLLATE expression on such data type, but it still has
a collation used.I don't think it's a bug. The information is available, but you have
to follow the graph to get it. Considering that the composite type
could be something like CREATE TYPE my_type AS (fr_name text COLLATE
"fr_CA", en_name text COLLATE "en_CA"), there is no single collation
you could put into pg_index.indcollation anyway. As for pg_depend,
it's currently enough for the index to depend on the type, and the
type to depend on the collation, because the purpose of dependencies
is to control dropping and dumping order, but for our new purpose we
also need to create direct dependencies index -> "fr_CA", index ->
"en_CA" so we can record the current versions.
Oh right, I didn't think about that.
3. Test t/002_pg_dump.pl in src/bin/pg_upgrade fails.
Apparently neither "make check" nor "make world" run this test :(
This was broken due collversion support in pg_dump, I have fixed it
locally.make check-world
Thanks!
Re: Collation versioning
On Fri, Nov 8, 2019 at 5:40 PM Laurenz Albe <laurenz.albe@cybertec.at> wrote:
On Fri, 2019-11-08 at 15:04 +1300, Thomas Munro wrote:
3. We don't know if pre-13 indexes are corrupted or not, and we'll
record that with a special value just as in proposal #1, except that
we could show a different hint for that special version value. It
would tell you can you can either REINDEX, or run ALTER INDEX ...
DEPENDS ON COLLATION "fr_FR" VERSION '34.0' if you believe the index
to have been created with the current collation version on an older
release of PostgreSQL that didn't track versions.
#3 is the best proposal, but there is still the need to run
ALTER INDEX on all affected indexes to keep PostgreSQL from nagging.
Perhaps the situation could be improved with a pg_upgrade option
--i-know-my-indexes-are-fine that causes a result like #2.
Together with a bold note in the release notes, this may relieve
the pain.
I suppose another reason to use such a switch would be if there is a
change in the versioning scheme; for example, as of today in master we
are using the glibc version, but a future glibc release might offer an
interface to query the CLDR version it's using, and then a future
release of PostgreSQL might get support for that, so the strings would
change between major version of PostgreSQL but you might want to be
able to tell pg_upgrade that your indexes are good.
Re: Collation versioning
On Tue, Nov 26, 2019 at 3:44 PM Thomas Munro <thomas.munro@gmail.com> wrote:
I suppose another reason to use such a switch would be if there is a
change in the versioning scheme; for example, as of today in master we
are using the glibc version, but a future glibc release might offer an
interface to query the CLDR version it's using, and then a future
release of PostgreSQL might get support for that, so the strings would
change between major version of PostgreSQL but you might want to be
able to tell pg_upgrade that your indexes are good.
Yeah, I like #3 too. If we're going to the trouble to build all of
this mechanism, it seems worth it to build the additional machinery to
be precise about the difference between "looks like a problem" and "we
don't know".
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: Collation versioning
On Wed, Nov 27, 2019 at 04:09:57PM -0500, Robert Haas wrote:
Yeah, I like #3 too. If we're going to the trouble to build all of
this mechanism, it seems worth it to build the additional machinery to
be precise about the difference between "looks like a problem" and "we
don't know".
Indeed, #3 sounds like a sensible way of doing things. The two others
may cause random problems which are harder to actually detect and act
on as we should avoid as much as possible a forced system-wide REINDEX
after an upgrade to a post-13 PG.
--
Michael
Re: Collation versioning
On Thu, Nov 28, 2019 at 5:50 AM Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Nov 27, 2019 at 04:09:57PM -0500, Robert Haas wrote:
Yeah, I like #3 too. If we're going to the trouble to build all of
this mechanism, it seems worth it to build the additional machinery to
be precise about the difference between "looks like a problem" and "we
don't know".Indeed, #3 sounds like a sensible way of doing things. The two others
may cause random problems which are harder to actually detect and act
on as we should avoid as much as possible a forced system-wide REINDEX
after an upgrade to a post-13 PG.
Thanks everyone for the feedback! Since there seems to be an
agreement on #3, here's a proposal.
What we could do is storing an empty string if the compatibility is
unknown, and detect it in index_check_collation_version() to report a
slightly different message. I'm assuming that not knowing the
compatibility would be system-wide rather than per collation, so we
could use an sql query like this:
ALTER INDEX idx_name DEPENDS ON COLLATION UNKNOWN VERSION
If adding (un)reserved keywords is not an issue, we could also instead
use something along ALTER INDEX idx_name DEPENDS ON ALL COLLATIONS
and/or ALL VERSIONS UNKNOWN, or switch to:
ALTER INDEX idx_name ALTER [ COLLATION coll_name | ALL COLLATIONS ]
DEPENDS ON [ UNKOWN VERSION | VERSION 'version_string' ]
Obviously, specific versions would require a specific collation, and
at least UNKNOWN VERSION would only be allowed in binary upgrade mode,
and not documented. I have also some other ideas for additional DDL
to let users deal with catalog update after a compatible collation
library upgrade, but let's deal with that later.
Then for pg_upgrade, we can add a --collations-are-binary-compatible
switch or similar:
If upgrading from pre-v13
- without the switch, we'd generate the VERSION UNKNOWN for all
indexes during pg_dump in upgrade_mode
- with the switch, do nothing as all indexes would already be
pointing to the currently installed version
If upgrading from post v13, the switch shouldn't be useful as versions
will be restored, and if there was a collation library upgrade it
should be handled manually, same as if such an upgrade is done without
pg_upgrade-ing the cluster. I'd personally disallow it to avoid users
to shoot themselves in the foot.
Does this sounds sensible?
Re: Collation versioning
On Thu, Nov 28, 2019 at 8:08 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
What we could do is storing an empty string if the compatibility is
unknown, and detect it in index_check_collation_version() to report a
slightly different message. I'm assuming that not knowing the
compatibility would be system-wide rather than per collation, so we
could use an sql query like this:ALTER INDEX idx_name DEPENDS ON COLLATION UNKNOWN VERSION
If adding (un)reserved keywords is not an issue, we could also instead
use something along ALTER INDEX idx_name DEPENDS ON ALL COLLATIONS
and/or ALL VERSIONS UNKNOWN, or switch to:
Adding unreserved keywords isn't a huge issue, but it's nicer to avoid
it if we can. Bloating the parser tables isn't that expensive, but
neither is it free. Maybe spell it like this:
ALTER INDEX idx_name DEPENDS ON COLLATION blah VERSION blah;
-- I care about collations and I know which one and which version.
ALTER INDEX idx_name DEPENDS ON SOME COLLATION;
-- I care about collations but I don't know which one.
ALTER INDEX idx_name DEPENDS ON NO COLLATION;
-- I don't care about collations at all.
-- Not sure if we need this.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: Collation versioning
On Mon, Dec 2, 2019 at 2:00 PM Robert Haas <robertmhaas@gmail.com> wrote:
ALTER INDEX idx_name DEPENDS ON COLLATION blah VERSION blah;
-- I care about collations and I know which one and which version.ALTER INDEX idx_name DEPENDS ON SOME COLLATION;
-- I care about collations but I don't know which one.
This seems a little bit ambiguous. I wouldn't expect this command to
trash all recorded versions given how it's spelled.
ALTER INDEX idx_name DEPENDS ON NO COLLATION;
-- I don't care about collations at all.
-- Not sure if we need this.
This should be an alias for "trust me all collation are ok"? It's
certainly useful for collation library upgrade that don't break
indexes, but I'd prefer to spell it something like "CURRENT VERSION".
I'll also work on that, but preferably in a later patch as there'll be
additional need (process all indexes with a given collation or
collation version for instance).
While looking at the list of keywords again, I think that "ANY" is a
better candidate:
ALTER INDEX idx_name DEPENDS ON [ ANY COLLATION | COLLATION blah ] [
UNKNOWN VERSION | VERSION blah ]
or
ALTER INDEX idx_name ALTER [ ANY COLLATION | COLLATION blah ] DEPENDS
ON [ UNKNOWN VERSION | VERSION blah ]
I like the 2nd one as it's more obvious that the command will only
modify existing dependencies.
Re: Collation versioning
On Tue, Dec 3, 2019 at 9:38 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Mon, Dec 2, 2019 at 2:00 PM Robert Haas <robertmhaas@gmail.com> wrote:
ALTER INDEX idx_name DEPENDS ON COLLATION blah VERSION blah;
-- I care about collations and I know which one and which version.ALTER INDEX idx_name DEPENDS ON SOME COLLATION;
-- I care about collations but I don't know which one.This seems a little bit ambiguous. I wouldn't expect this command to
trash all recorded versions given how it's spelled.ALTER INDEX idx_name DEPENDS ON NO COLLATION;
-- I don't care about collations at all.
-- Not sure if we need this.This should be an alias for "trust me all collation are ok"? It's
certainly useful for collation library upgrade that don't break
indexes, but I'd prefer to spell it something like "CURRENT VERSION".
I'll also work on that, but preferably in a later patch as there'll be
additional need (process all indexes with a given collation or
collation version for instance).While looking at the list of keywords again, I think that "ANY" is a
better candidate:ALTER INDEX idx_name DEPENDS ON [ ANY COLLATION | COLLATION blah ] [
UNKNOWN VERSION | VERSION blah ]
or
ALTER INDEX idx_name ALTER [ ANY COLLATION | COLLATION blah ] DEPENDS
ON [ UNKNOWN VERSION | VERSION blah ]I like the 2nd one as it's more obvious that the command will only
modify existing dependencies.
I'm attaching v4 versions, rebased and with the following modifications:
- I implemented the ALTER INDEX name DEPENDS ON [ ANY COLLATION |
COLLATION name ] [ UNKNOWN VERSION | VERSION blah ] syntax. Both are
still only allowed in binary upgrade mode, so I didn't add
documentation for those, or psql tab completion.
- Those commands now requires the collation name rather than oid.
This will be helpful if we want to make the commands above available
for users, or some similar commands like marking an index depending on
the current version for some specific collation
- added the type regcollation to ease pg_dump work
- unknown collation version is handled with a recorded empty string,
and lead to a slightly different message
- new "--collation-binary-compatible" documented option for
pg_upgrade, leading to using a new and undocumented
"--unknown-collations-binary-compatible" option for pg_dump. The new
option in pg_dump only works in binary upgrade mode, and only prevents
outputting the ALTER INDEX ... UNKNOWN VERSION commands. Known
existing versions are always preserved.
- dependencies for composite and other exotic types for regular index
columns should now be correctly handled
- dependencies for index expression and predicate will now track
default collation, with additional support for composite types
- to do so, I had to handle the collation version retrieval in
recordMultipleDependencies() using a new "track_version" bool
parameter, passed around in some functions. This seemed the best way
to do so, as this is the common point in
recordDependencyOnSingleRelExpr(), index_create() and others.
The spurious messages when issuing a REINDEX is still not fixed. I'm
also attaching an sql file that I used to check all cases of
non-trivial collation dependencies I could think of.
Attachments:
0002-Add-pg_depend.refobjversion-v4.patchapplication/octet-stream; name=0002-Add-pg_depend.refobjversion-v4.patchDownload
From 233438fcb3e5db5cceacaa481f94a01dc2802b45 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 20 +++++++++--
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
6 files changed, 56 insertions(+), 30 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index d07bb4496e..0bb1504b20 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2669,7 +2672,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index a060c25d2e..f1dc1143a7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -101,9 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ values[Anum_pg_depend_refobjversion - 1] = version ? NameGetDatum(version) : CStringGetDatum("");
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 1f6d8939be..0909723aaf 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1562,55 +1562,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ff50d594f6..b5d3276e77 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index f786445fb2..e30896178a 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -55,6 +55,7 @@ CATALOG(pg_depend,2608,DependRelationId)
Oid refclassid; /* OID of table containing object */
Oid refobjid; /* OID of object itself */
int32 refobjsubid; /* column number, or 0 if not used */
+ NameData refobjversion; /* version tracking, or empty if not used */
/*
* Precise semantics of the relationship are specified by the deptype
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..44d5e756a3 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | refobjversion | deptype
+---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
0001-Remove-pg_collation.collversion-v4.patchapplication/octet-stream; name=0001-Remove-pg_collation.collversion-v4.patchDownload
From b59476eb3d85a9359dbc4244aef3d6216163b330 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/5] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
18 files changed, 11 insertions(+), 290 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 55694c4368..21adb1640a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2127,17 +2127,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 57a1539506..22af58b262 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21655,10 +21655,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index dd99d53547..e67db0d291 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 919e092483..4f1442df03 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index a74b56bb59..ebba5d4fa9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3177,16 +3177,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5166,9 +5156,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 2fcd4a3467..aa786b563e 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1101,14 +1101,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3262,9 +3254,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c5086846de..8ba21cefcf 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -827,7 +827,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10298,21 +10297,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 3a03ca7e2f..5a67901e21 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1685,10 +1685,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2789,10 +2785,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3401,10 +3393,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index fcdbaae37b..b7a4a1421e 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 08658c8e86..cd40a55845 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13485,7 +13485,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13557,7 +13561,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 367ce3607b..7f88a41ad4 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index d3366f361d..44c3b5fd85 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index cc5dfed0bf..1fd27d5fd3 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index e25f5d50b3..8090a5cb0c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ff626cbe61..6b4e188e81 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1850,17 +1850,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
0005-Preserve-index-dependencies-on-collation-during-pg_u-v4.patchapplication/octet-stream; name=0005-Preserve-index-dependencies-on-collation-during-pg_u-v4.patchDownload
From 6523eb5a00c5d59eb416c24d8eb7eba471178ced Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/5] Preserve index dependencies on collation during
pg_upgrade
Two new commands are added.
ALTER INDEX i_name DEPENDS ON COLLATION c_name VERSION v_name
to force a dependency on a collation specific version, and
ALTER INDEX i_name DEPENDS ON ANY COLLATION UNKOWN VERSION
to specify that the version is unknown for all collations on a specific index.
Both commands are only allowed in binary upgrade mode. Also teach pg_dump to
emit such commands for all indexes, including indexes created for constraints,
when run with --binary-upgrade flag.
When pg_upgrade is run against an older version, collation versions are not
known and pg_dump will by default emit an alter index command to mark all
collation versions as unkown. However, it's possible that pg_upgrade is run
without upgrading the underlying collation libraries, so a new option
--collation-binary-compatible is added to avoid this behavior. This will
result in running pg_dump with a new --unknown-collations-binary-compatible
option, that can only be used in binary upgrade mode, to prevent pg_dump from
emitting the alter index commands if the dependent collation version is
unknown. Note that if the collation version is known, this flag won't change
the behavior and the previous collation version will be preserved.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 16 +++
src/backend/catalog/index.c | 11 ++
src/backend/commands/tablecmds.c | 72 +++++++++++++
src/backend/nodes/copyfuncs.c | 2 +
src/backend/parser/gram.y | 18 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 178 +++++++++++++++++++++++++++++--
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 ++
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/nodes/parsenodes.h | 5 +-
12 files changed, 306 insertions(+), 13 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index d4da566d76..40e8a5e919 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,22 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ When upgrading from a PostgreSQL major version 12 or older, all indexes
+ will be marked as depending on an unknown collation version, as such
+ versions weren't tracked. As a result, numerous warning messages will
+ be emitted as it can be a sign of a corrupted index. If you're not
+ upgrading the collation libraries, and if you're absolutly certain that
+ all existing indexes are compatible with the current collation
+ libraries, you can use this flag to change this behavior and mark all
+ indexes as depending on current collation libraries version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index a0977df7db..7967a5d6b0 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1252,7 +1252,18 @@ index_check_collation_version(const ObjectAddress *otherObject,
/* Compare with the current version. */
get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+
if (strncmp(NameStr(*version),
+ "",
+ sizeof(NameData)) == 0)
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unkown version, but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ else if (strncmp(NameStr(*version),
NameStr(current_version),
sizeof(NameData)) != 0)
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 5440eb9015..bb6b996ae3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -201,6 +201,13 @@ typedef struct NewColumnValue
ExprState *exprstate; /* execution state */
} NewColumnValue;
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ NameData version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/*
* Error-reporting support for RemoveRelations
*/
@@ -529,6 +536,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecDependsOnCollationVersion(Relation rel, List *coll,
+ char *version);
/* ----------------------------------------------------------------
@@ -3806,6 +3815,11 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ /* Only used in binary upgrade mode */
+ case AT_DependsOnCollationVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -3959,6 +3973,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
+ * [UNKNOWN VERSION | VERSION ...] */
+ if (!IsBinaryUpgrade)
+ ereport(ERROR,
+ (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ (errmsg("command can only be called when server is in binary upgrade mode"))));
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode);
@@ -4483,6 +4507,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_DependsOnCollationVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecDependsOnCollationVersion(rel, cmd->object, cmd->version);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -16732,3 +16761,46 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+static NameData *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return &forced_dependency->version;
+}
+
+static void
+ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ if (coll == NIL)
+ forced_dependency.oid = InvalidOid;
+ else
+ forced_dependency.oid = get_collation_oid(coll, false);
+ strncpy(NameStr(forced_dependency.version), version, sizeof(NameData));
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ebba5d4fa9..f63792500a 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3168,7 +3168,9 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
+ COPY_STRING_FIELD(version);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
COPY_SCALAR_FIELD(behavior);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8ba21cefcf..4f53737e66 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2549,6 +2549,24 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> DEPENDS ON COLLATION ... VERSION ... */
+ | DEPENDS ON COLLATION any_name VERSION_P Sconst
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = $4;
+ n->version = $6;
+ $$ = (Node *)n;
+ }
+ /* ALTER INDEX <name> DEPENDS ON ANY COLLATION UNKNOWN VERSION */
+ | DEPENDS ON ANY COLLATION UNKNOWN VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = NIL;
+ n->version = "";
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index cd40a55845..f6178ba152 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -288,6 +288,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -388,6 +390,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -708,6 +711,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6841,7 +6848,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6877,7 +6886,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid::pg_catalog.regcollation ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = 1259 AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = 3456 AND "
+ " refobjversion != '') AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(refobjversion ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = 1259 AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = 3456 AND "
+ " refobjversion != '') AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6902,7 +6966,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "'' AS inddependoids, "
+ "'' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6941,7 +7007,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'' AS inddependoids, "
+ "'' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6976,7 +7044,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'' AS inddependoids, "
+ "'' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7007,7 +7077,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'' AS inddependoids, "
+ "'' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7041,7 +7113,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'' AS inddependoids, "
+ "'' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7081,6 +7155,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7106,6 +7182,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16374,10 +16452,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16437,6 +16516,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16466,6 +16549,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18423,6 +18521,64 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of ALTER INDEX ... DEPENDS ON COLLATION ... VERSION ....
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, "") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, "") == 0);
+
+ appendPQExpBuffer(buffer, "ALTER INDEX %s",
+ fmtQualifiedDumpable(indxinfo));
+ appendPQExpBuffer(buffer, " DEPENDS ON ANY COLLATION UNKNOWN VERSION\n");
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ appendPQExpBuffer(buffer, "ALTER INDEX %s",
+ fmtQualifiedDumpable(indxinfo));
+ appendPQExpBuffer(buffer, " DEPENDS ON COLLATION %s VERSION ",
+ inddependoidsarray[i]);
+ appendStringLiteral(buffer, inddependversionsarray[i], enc, true);
+ appendPQExpBuffer(buffer, "\n");
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 7b2c1524a5..6f67d195dd 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index bdb5006fa6..bc9480d8cf 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 14351e8028..ffa3394cc5 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 729f86aa32..202a8b67bd 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -294,6 +294,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 6b4e188e81..4e559fea09 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1824,7 +1824,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_DependsOnCollationVersion /* DEPENDS ON COLLATION ... VERSION ... */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1840,8 +1841,10 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
+ char *version; /* version reference for collation dependency */
RoleSpec *newowner;
Node *def; /* definition of new column, index,
* constraint, or parent table */
--
2.20.1
0004-Implement-type-regcollation-v4.patchapplication/octet-stream; name=0004-Implement-type-regcollation-v4.patchDownload
From bc56c34603eb827aa4d52ec0da3241e48ef2a40d Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH 4/5] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
4 files changed, 185 insertions(+)
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index 4b1decf81e..7ced3f400d 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index aabfa7af03..e4c7dec4d9 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ac8f64b219..aef574c46c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6636,6 +6636,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7422,6 +7431,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index d9b35af914..5d0b077e4a 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
0003-Track-collation-versions-for-indexes-v4.patchapplication/octet-stream; name=0003-Track-collation-versions-for-indexes-v4.patchDownload
From 6faee17b28f9342dbf20535a8708f02c869f4991 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. Whenever we load an index into the
relcache, check if the collation versions still match those reported
by the collation provider. Warn that the index may be corrupted if
not.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 143 ++++++++++++++++++---
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 116 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 69 +++++++---
src/backend/catalog/pg_type.c | 47 +++++++
src/backend/utils/adt/pg_locale.c | 39 ++++++
src/backend/utils/cache/relcache.c | 5 +
src/include/catalog/dependency.h | 21 ++-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 1 +
src/test/regress/expected/create_index.out | 8 +-
13 files changed, 402 insertions(+), 60 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 0bb1504b20..e8e3129886 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -137,6 +137,8 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
} find_expr_references_context;
/*
@@ -437,6 +439,63 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ NameData *new_version;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ new_version = callback(&otherObject, &foundDep->refobjversion,
+ userdata);
+ if (new_version)
+ {
+ /* Make a modifyable copy. */
+ tup = heap_copytuple(tup);
+ foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ foundDep->refobjversion = *new_version;
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1602,9 +1661,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1691,14 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1752,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1774,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1775,6 +1839,32 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+ /* and its type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) && (coll != DEFAULT_COLLATION_OID ||
+ context->track_version))
+ add_object_address(OCLASS_COLLATION, lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
}
else if (rte->rtekind == RTE_JOIN)
{
@@ -1808,11 +1898,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1902,7 +1994,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1990,7 +2083,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2021,7 +2115,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2034,7 +2129,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2047,7 +2143,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2136,7 +2233,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2254,7 +2352,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2276,7 +2376,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2672,8 +2774,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 8404904710..f9d9ef96c8 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2273,7 +2273,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2283,7 +2283,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3527,7 +3527,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e9955707fa..a0977df7db 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1116,19 +1117,26 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /*
+ * Store dependency on collations The C and posix collations are
+ * pinned, so don't bother recording them
+ */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ List *collations = NIL;
+
+ if (OidIsValid(collationObjectId[i]))
+ collations = list_make1_oid(collationObjectId[i]);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ collations = GetTypeCollations(att->atttypid);
}
+
+ recordDependencyOnCollations(&myself, collations);
}
/* Store dependency on operator classes */
@@ -1148,7 +1156,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
/* Store dependencies on anything mentioned in predicate */
@@ -1158,7 +1166,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1230,6 +1238,91 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static NameData *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ NameData current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+ if (strncmp(NameStr(*version),
+ NameStr(current_version),
+ sizeof(NameData)) != 0)
+ {
+ if (strncmp(NameStr(*version),
+ "",
+ sizeof(NameData)) == 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unkown version, but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ NameStr(*version),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static NameData *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NameData *current_version = (NameData *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ get_collation_version_for_oid(otherObject->objectId, current_version);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3597,6 +3690,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 56568b0105..a271eb2e2d 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -361,7 +361,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f1dc1143a7..d78df96304 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -26,6 +27,7 @@
#include "miscadmin.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
@@ -44,22 +46,29 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, true);
+ }
}
/*
@@ -69,9 +78,9 @@ recordDependencyOnVersion(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +88,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ NameData version;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -99,12 +109,37 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ version.data[0] = '\0';
+
+ if (track_version)
+ {
+ /* Only dependency on a collation needs to be tracked */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;;
+ get_collation_version_for_oid(referenced->objectId, &version);
+ }
+ }
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -117,7 +152,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
- values[Anum_pg_depend_refobjversion - 1] = version ? NameGetDatum(version) : CStringGetDatum("");
+ values[Anum_pg_depend_refobjversion - 1] = NameGetDatum(&version);
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index a8c1de511f..818c53d7e4 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,52 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all collations that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ result = lappend_oid(result, typeTup->typcollation);
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ result = lappend_oid(result, att->attcollation);
+ else
+ result = list_concat(result, GetTypeCollations(att->atttypid));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat(result, GetTypeCollations(typeTup->typbasetype));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat(result, GetTypeCollations(typeTup->typelem));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index b7a4a1421e..19a1a6573a 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -1501,6 +1503,43 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collatoin OID.
+ */
+void
+get_collation_version_for_oid(Oid oid, NameData *output)
+{
+ HeapTuple tp;
+ const char *version;
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ memset(output, 0, sizeof(NameData));
+ if (version)
+ strncpy(NameStr(*output), version, sizeof(NameData));
+ ReleaseSysCache(tp);
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 50f8912c13..f1b3d50d1f 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -1469,6 +1470,10 @@ RelationInitIndexAccessInfo(Relation relation)
indcoll = (oidvector *) DatumGetPointer(indcollDatum);
memcpy(relation->rd_indcollation, indcoll->values, indnkeyatts * sizeof(Oid));
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsCatalogRelation(relation))
+ index_check_collation_versions(RelationGetRelid(relation));
+
/*
* indclass cannot be referenced directly through the C struct, because it
* comes after the variable-width indkey field. Must extract the datum
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b5d3276e77..eb34616635 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,28 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef NameData *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 27d9e537d3..f8b7a5ed54 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 0c273a0449..a38fd566ff 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index b4b3aa5843..023ef693e1 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern void get_collation_version_for_oid(Oid collid, NameData *output);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 1cdb7a9663..2f30d66935 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -1990,15 +1990,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2018,15 +2020,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
--
2.20.1
Re: Collation versioning
On Fri, Nov 8, 2019 at 2:24 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Thu, Nov 7, 2019 at 10:20 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
I didn't do anything for the spurious warning when running a reindex,
and kept original approach of pg_depend catalog.I see three options:
1. Change all or some of index_open(), relation_open(),
RelationIdGetRelation(), RelationBuildDesc() and
RelationInitIndexAccessInfo() to take some kind of flag so we can say
NO_COLLATION_VERSION_CHECK_PLEASE, and then have ReindexIndex() pass
that flag down when opening it for the purpose of rebuilding it.
2. Use a global state to suppress these warnings while opening that
index. Perhaps ReindexIndex() would call RelCacheWarnings(false)
before index_open(), and use PG_FINALLY to make sure it restores
warnings with RelCacheWarnings(true). (This is a less-code-churn
bad-programming-style version of #1.)
3. Move the place that we run the collation version check. Instead
of doing it in RelationInitIndexAccessInfo() so that it runs when the
relation cache first loads the index, put it into a new routine
RelationCheckValidity() and call that from ... I don't know, some
other place that runs whenever we access indexes but not when we
rebuild them.3's probably a better idea, if you can find a reasonable place to call
it from. I'm thinking some time around the time the executor acquires
locks, but using a flag in the relcache entry to make sure it doesn't
run the check again if there were no warnings last time (so one
successful version check turns the extra work off for the rest of this
relcache entry's lifetime). I think it'd be a feature, not a bug, if
it gave you the warning every single time you executed a query using
an index that has a mismatch... but a practical alternative would be
to check only once per index and that probably makes more sense.
I tried to dig into approach #3. I think that trying to perform this
check when the executor acquires locks won't work well with at least
with partitioned table. I instead tried to handle that in
get_relation_info(), adding a flag in the relcache to emit each
warning only once per backend lifetime, and this seems to work quite
well.
I think that on top of that, we should make sure that non-full vacuum
and analyze also emit such warnings, so that autovacuum can spam the
logs too.
Re: Collation versioning
On Fri, Dec 6, 2019 at 10:29 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
I'm attaching v4 versions, rebased and with the following modifications:
- I implemented the ALTER INDEX name DEPENDS ON [ ANY COLLATION |
COLLATION name ] [ UNKNOWN VERSION | VERSION blah ] syntax. Both are
still only allowed in binary upgrade mode, so I didn't add
documentation for those, or psql tab completion.
- Those commands now requires the collation name rather than oid.
This will be helpful if we want to make the commands above available
for users, or some similar commands like marking an index depending on
the current version for some specific collation
- added the type regcollation to ease pg_dump work
- unknown collation version is handled with a recorded empty string,
and lead to a slightly different message
- new "--collation-binary-compatible" documented option for
pg_upgrade, leading to using a new and undocumented
"--unknown-collations-binary-compatible" option for pg_dump. The new
option in pg_dump only works in binary upgrade mode, and only prevents
outputting the ALTER INDEX ... UNKNOWN VERSION commands. Known
existing versions are always preserved.
- dependencies for composite and other exotic types for regular index
columns should now be correctly handled
- dependencies for index expression and predicate will now track
default collation, with additional support for composite types
- to do so, I had to handle the collation version retrieval in
recordMultipleDependencies() using a new "track_version" bool
parameter, passed around in some functions. This seemed the best way
to do so, as this is the common point in
recordDependencyOnSingleRelExpr(), index_create() and others.The spurious messages when issuing a REINDEX is still not fixed. I'm
also attaching an sql file that I used to check all cases of
non-trivial collation dependencies I could think of.
Hearing no objection in [1]/messages/by-id/CAOBaU_YbCQ6=8_VOdZY0v-cXX6+BkgScpNRmRjJzESdzv1SZMA@mail.gmail.com, attached v5 with following changes:
- fix the spurious warnings by doing the version check in
get_relation_info and vacuum_open_relation, and add a flag in
RelationData to mark the check as already being done
- change the IsCatalogRelation() check to IsSystemRelation() to also
ignore toast indexes, as those can also be safely ignored too
- add a new ALTER INDEX idxname DEPENDS ON COLLATION collname CURRENT
VERSION to let users remove the warnings for safe library upgrade.
Documentation and tab completion added
If I'm not mistaken, all issues I was aware of are now fixed.
[1]: /messages/by-id/CAOBaU_YbCQ6=8_VOdZY0v-cXX6+BkgScpNRmRjJzESdzv1SZMA@mail.gmail.com
Attachments:
0003-Track-collation-versions-for-indexes-v5.patchapplication/octet-stream; name=0003-Track-collation-versions-for-indexes-v5.patchDownload
From 448acf7ff91debd3a8aa2f1bb1785d20fe1d3555 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 3/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 143 ++++++++++++++++++---
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 101 +++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 69 +++++++---
src/backend/catalog/pg_type.c | 47 +++++++
src/backend/commands/vacuum.c | 31 +++++
src/backend/optimizer/util/plancat.c | 9 ++
src/backend/utils/adt/pg_locale.c | 39 ++++++
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 21 ++-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 1 +
src/include/utils/rel.h | 1 +
src/test/regress/expected/create_index.out | 8 +-
16 files changed, 425 insertions(+), 60 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 0bb1504b20..e8e3129886 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -137,6 +137,8 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
} find_expr_references_context;
/*
@@ -437,6 +439,63 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ NameData *new_version;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ new_version = callback(&otherObject, &foundDep->refobjversion,
+ userdata);
+ if (new_version)
+ {
+ /* Make a modifyable copy. */
+ tup = heap_copytuple(tup);
+ foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ foundDep->refobjversion = *new_version;
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1602,9 +1661,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1691,14 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1752,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1774,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1775,6 +1839,32 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+ /* and its type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) && (coll != DEFAULT_COLLATION_OID ||
+ context->track_version))
+ add_object_address(OCLASS_COLLATION, lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
}
else if (rte->rtekind == RTE_JOIN)
{
@@ -1808,11 +1898,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1902,7 +1994,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1990,7 +2083,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2021,7 +2115,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2034,7 +2129,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2047,7 +2143,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2136,7 +2233,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2254,7 +2352,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2276,7 +2376,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2672,8 +2774,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 8404904710..f9d9ef96c8 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2273,7 +2273,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2283,7 +2283,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3527,7 +3527,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e9955707fa..dfc0ecbf3b 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1116,19 +1117,26 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /*
+ * Store dependency on collations The C and posix collations are
+ * pinned, so don't bother recording them
+ */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ List *collations = NIL;
+
+ if (OidIsValid(collationObjectId[i]))
+ collations = list_make1_oid(collationObjectId[i]);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ collations = GetTypeCollations(att->atttypid);
}
+
+ recordDependencyOnCollations(&myself, collations);
}
/* Store dependency on operator classes */
@@ -1148,7 +1156,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
/* Store dependencies on anything mentioned in predicate */
@@ -1158,7 +1166,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1230,6 +1238,76 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static NameData *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ NameData current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+ if (strncmp(NameStr(*version),
+ NameStr(current_version),
+ sizeof(NameData)) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ NameStr(*version),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static NameData *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NameData *current_version = (NameData *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ get_collation_version_for_oid(otherObject->objectId, current_version);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3597,6 +3675,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 56568b0105..a271eb2e2d 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -361,7 +361,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f1dc1143a7..d78df96304 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -26,6 +27,7 @@
#include "miscadmin.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
@@ -44,22 +46,29 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, true);
+ }
}
/*
@@ -69,9 +78,9 @@ recordDependencyOnVersion(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +88,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ NameData version;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -99,12 +109,37 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ version.data[0] = '\0';
+
+ if (track_version)
+ {
+ /* Only dependency on a collation needs to be tracked */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;;
+ get_collation_version_for_oid(referenced->objectId, &version);
+ }
+ }
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -117,7 +152,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
- values[Anum_pg_depend_refobjversion - 1] = version ? NameGetDatum(version) : CStringGetDatum("");
+ values[Anum_pg_depend_refobjversion - 1] = NameGetDatum(&version);
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index a8c1de511f..818c53d7e4 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,52 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all collations that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ result = lappend_oid(result, typeTup->typcollation);
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ result = lappend_oid(result, att->attcollation);
+ else
+ result = list_concat(result, GetTypeCollations(att->atttypid));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat(result, GetTypeCollations(typeTup->typbasetype));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat(result, GetTypeCollations(typeTup->typelem));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index da1da23400..d0841ea657 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -579,6 +581,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 5e889d1861..7e10568953 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index b7a4a1421e..59b46fcf52 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -1501,6 +1503,43 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ */
+void
+get_collation_version_for_oid(Oid oid, NameData *output)
+{
+ HeapTuple tp;
+ const char *version;
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ memset(output, 0, sizeof(NameData));
+ if (version)
+ strncpy(NameStr(*output), version, sizeof(NameData));
+ ReleaseSysCache(tp);
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 50f8912c13..4fed56a17c 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5637,6 +5638,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b5d3276e77..eb34616635 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,28 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef NameData *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 27d9e537d3..f8b7a5ed54 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 0c273a0449..a38fd566ff 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index b4b3aa5843..023ef693e1 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern void get_collation_version_for_oid(Oid collid, NameData *output);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 31d8a1a10e..c6ed5d3e5a 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -62,6 +62,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 645ae2cf34..46c0820eb3 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2003,15 +2003,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2031,15 +2033,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
--
2.20.1
0001-Remove-pg_collation.collversion-v5.patchapplication/octet-stream; name=0001-Remove-pg_collation.collversion-v5.patchDownload
From 37ad7d923b0d46e5d2aeda0aaa70193cfc150d00 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/6] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
18 files changed, 11 insertions(+), 290 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 55694c4368..21adb1640a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2127,17 +2127,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 57a1539506..22af58b262 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21655,10 +21655,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index dd99d53547..e67db0d291 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 919e092483..4f1442df03 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index a74b56bb59..ebba5d4fa9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3177,16 +3177,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5166,9 +5156,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 2fcd4a3467..aa786b563e 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1101,14 +1101,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3262,9 +3254,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c5086846de..8ba21cefcf 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -827,7 +827,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10298,21 +10297,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 3a03ca7e2f..5a67901e21 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1685,10 +1685,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2789,10 +2785,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3401,10 +3393,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index fcdbaae37b..b7a4a1421e 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 08658c8e86..cd40a55845 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13485,7 +13485,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13557,7 +13561,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 367ce3607b..7f88a41ad4 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index d3366f361d..44c3b5fd85 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index cc5dfed0bf..1fd27d5fd3 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index e25f5d50b3..8090a5cb0c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ff626cbe61..6b4e188e81 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1850,17 +1850,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
0005-Preserve-index-dependencies-on-collation-during-pg_u-v5.patchapplication/octet-stream; name=0005-Preserve-index-dependencies-on-collation-during-pg_u-v5.patchDownload
From a226426df2affc26a6db7c705cb85173641cee86 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/6] Preserve index dependencies on collation during
pg_upgrade
Two new commands are added.
ALTER INDEX i_name DEPENDS ON COLLATION c_name VERSION v_name
to force a dependency on a collation specific version, and
ALTER INDEX i_name DEPENDS ON ANY COLLATION UNKOWN VERSION
to specify that the version is unknown for all collations on a specific index.
Both commands are only allowed in binary upgrade mode. Also teach pg_dump to
emit such commands for all indexes, including indexes created for constraints,
when run with --binary-upgrade flag.
When pg_upgrade is run against an older version, collation versions are not
known and pg_dump will by default emit an alter index command to mark all
collation versions as unkown. However, it's possible that pg_upgrade is run
without upgrading the underlying collation libraries, so a new option
--collation-binary-compatible is added to avoid this behavior. This will
result in running pg_dump with a new --unknown-collations-binary-compatible
option, that can only be used in binary upgrade mode, to prevent pg_dump from
emitting the alter index commands if the dependent collation version is
unknown. Note that if the collation version is known, this flag won't change
the behavior and the previous collation version will be preserved.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 16 +++
src/backend/catalog/index.c | 16 ++-
src/backend/commands/tablecmds.c | 75 +++++++++++++
src/backend/nodes/copyfuncs.c | 2 +
src/backend/parser/gram.y | 18 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 179 +++++++++++++++++++++++++++++--
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 ++
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/nodes/parsenodes.h | 5 +-
12 files changed, 314 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index d4da566d76..40e8a5e919 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,22 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ When upgrading from a PostgreSQL major version 12 or older, all indexes
+ will be marked as depending on an unknown collation version, as such
+ versions weren't tracked. As a result, numerous warning messages will
+ be emitted as it can be a sign of a corrupted index. If you're not
+ upgrading the collation libraries, and if you're absolutly certain that
+ all existing indexes are compatible with the current collation
+ libraries, you can use this flag to change this behavior and mark all
+ indexes as depending on current collation libraries version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index dfc0ecbf3b..adbf0a8131 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1252,11 +1252,24 @@ index_check_collation_version(const ObjectAddress *otherObject,
/* Compare with the current version. */
get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+
if (strncmp(NameStr(*version),
NameStr(current_version),
sizeof(NameData)) != 0)
{
- ereport(WARNING,
+ if (strncmp(NameStr(*version), "", sizeof(NameData)) == 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else
+ {
+ ereport(WARNING,
(errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
get_rel_name(relid),
get_collation_name(otherObject->objectId),
@@ -1264,6 +1277,7 @@ index_check_collation_version(const ObjectAddress *otherObject,
NameStr(current_version)),
errdetail("The index may be corrupted due to changes in sort order."),
errhint("REINDEX to avoid the risk of corruption.")));
+ }
}
return NULL;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index daa80ec4aa..67d1fd5732 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -201,6 +201,13 @@ typedef struct NewColumnValue
ExprState *exprstate; /* execution state */
} NewColumnValue;
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ NameData version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/*
* Error-reporting support for RemoveRelations
*/
@@ -527,6 +534,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecDependsOnCollationVersion(Relation rel, List *coll,
+ char *version);
/* ----------------------------------------------------------------
@@ -3804,6 +3813,11 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ /* Only used in binary upgrade mode */
+ case AT_DependsOnCollationVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -3957,6 +3971,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
+ * [UNKNOWN VERSION | VERSION ...] */
+ if (!IsBinaryUpgrade)
+ ereport(ERROR,
+ (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ (errmsg("command can only be called when server is in binary upgrade mode"))));
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode);
@@ -4481,6 +4505,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_DependsOnCollationVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecDependsOnCollationVersion(rel, cmd->object, cmd->version);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -16703,3 +16732,49 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+static NameData *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return &forced_dependency->version;
+}
+
+static void
+ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ if (coll == NIL)
+ forced_dependency.oid = InvalidOid;
+ else
+ forced_dependency.oid = get_collation_oid(coll, false);
+ strncpy(NameStr(forced_dependency.version), version, sizeof(NameData));
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ebba5d4fa9..f63792500a 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3168,7 +3168,9 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
+ COPY_STRING_FIELD(version);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
COPY_SCALAR_FIELD(behavior);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8ba21cefcf..4f53737e66 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2549,6 +2549,24 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> DEPENDS ON COLLATION ... VERSION ... */
+ | DEPENDS ON COLLATION any_name VERSION_P Sconst
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = $4;
+ n->version = $6;
+ $$ = (Node *)n;
+ }
+ /* ALTER INDEX <name> DEPENDS ON ANY COLLATION UNKNOWN VERSION */
+ | DEPENDS ON ANY COLLATION UNKNOWN VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = NIL;
+ n->version = "";
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index cd40a55845..dd32922775 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -288,6 +289,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -388,6 +391,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -708,6 +712,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6841,7 +6849,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6877,7 +6887,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid::pg_catalog.regcollation ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion != '') AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(refobjversion ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion != '') AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6902,7 +6967,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6941,7 +7008,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6976,7 +7045,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7007,7 +7078,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7041,7 +7114,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7081,6 +7156,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7106,6 +7183,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16374,10 +16453,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16437,6 +16517,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16466,6 +16550,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18423,6 +18522,64 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of ALTER INDEX ... DEPENDS ON COLLATION ... VERSION ....
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, "") == 0);
+
+ appendPQExpBuffer(buffer, "ALTER INDEX %s",
+ fmtQualifiedDumpable(indxinfo));
+ appendPQExpBuffer(buffer, " DEPENDS ON ANY COLLATION UNKNOWN VERSION\n");
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ appendPQExpBuffer(buffer, "ALTER INDEX %s",
+ fmtQualifiedDumpable(indxinfo));
+ appendPQExpBuffer(buffer, " DEPENDS ON COLLATION %s VERSION ",
+ inddependoidsarray[i]);
+ appendStringLiteral(buffer, inddependversionsarray[i], enc, true);
+ appendPQExpBuffer(buffer, "\n");
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 7b2c1524a5..6f67d195dd 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index bdb5006fa6..bc9480d8cf 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 14351e8028..ffa3394cc5 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 729f86aa32..202a8b67bd 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -294,6 +294,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 6b4e188e81..4e559fea09 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1824,7 +1824,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_DependsOnCollationVersion /* DEPENDS ON COLLATION ... VERSION ... */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1840,8 +1841,10 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
+ char *version; /* version reference for collation dependency */
RoleSpec *newowner;
Node *def; /* definition of new column, index,
* constraint, or parent table */
--
2.20.1
0002-Add-pg_depend.refobjversion-v5.patchapplication/octet-stream; name=0002-Add-pg_depend.refobjversion-v5.patchDownload
From 473b360079cb2bd5dfefa186a3a0d4c8e98ad565 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 20 +++++++++--
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
6 files changed, 56 insertions(+), 30 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index d07bb4496e..0bb1504b20 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2669,7 +2672,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index a060c25d2e..f1dc1143a7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -101,9 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ values[Anum_pg_depend_refobjversion - 1] = version ? NameGetDatum(version) : CStringGetDatum("");
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 1f6d8939be..0909723aaf 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1562,55 +1562,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ff50d594f6..b5d3276e77 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index f786445fb2..e30896178a 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -55,6 +55,7 @@ CATALOG(pg_depend,2608,DependRelationId)
Oid refclassid; /* OID of table containing object */
Oid refobjid; /* OID of object itself */
int32 refobjsubid; /* column number, or 0 if not used */
+ NameData refobjversion; /* version tracking, or empty if not used */
/*
* Precise semantics of the relationship are specified by the deptype
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..44d5e756a3 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | refobjversion | deptype
+---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
0004-Implement-type-regcollation-v5.patchapplication/octet-stream; name=0004-Implement-type-regcollation-v5.patchDownload
From ae93c4176b24f6dcd5d7baae28f7ebbaedec29e5 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH 4/6] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
4 files changed, 185 insertions(+)
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index 4b1decf81e..7ced3f400d 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index aabfa7af03..e4c7dec4d9 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ac8f64b219..aef574c46c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6636,6 +6636,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7422,6 +7431,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index d9b35af914..5d0b077e4a 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
0006-Add-a-new-ALTER-INDEX-name-DEPENDS-ON-COLLATION-name-v5.patchapplication/octet-stream; name=0006-Add-a-new-ALTER-INDEX-name-DEPENDS-ON-COLLATION-name-v5.patchDownload
From 1904912824a577cb87beea5d271285e1f045eae8 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH 6/6] Add a new ALTER INDEX name DEPENDS ON COLLATION name
CURRENT VERSION
This command allows privileged users to specifify that the currently installed
collation library version, for a specific collation, is binary compatible with
the one that was installed when the specified index was built for the last
time.
---
doc/src/sgml/ref/alter_index.sgml | 13 +++++++++++++
src/backend/commands/tablecmds.c | 14 ++++++++++++--
src/backend/parser/gram.y | 9 +++++++++
src/bin/psql/tab-complete.c | 26 +++++++++++++++++++++++++-
src/include/nodes/parsenodes.h | 5 ++++-
5 files changed, 63 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..db5ade29d1 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON COLLATION <replaceable class="parameter">collation_name</replaceable> CURRENT VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,18 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>DEPENDS ON EXTENSION</literal></term>
+ <listitem>
+ <para>
+ This form update the index existing dependency on a specific collation,
+ to specificy the the currently installed collation version is compatible
+ with the version used the last time the index was built. Be aware that
+ an incorrect use of this form can hide a corruption on the index.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 67d1fd5732..5a7da64f95 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -92,6 +92,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -3973,7 +3974,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
break;
case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
* [UNKNOWN VERSION | VERSION ...] */
- if (!IsBinaryUpgrade)
+ if (!IsBinaryUpgrade && cmd->version)
ereport(ERROR,
(errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
(errmsg("command can only be called when server is in binary upgrade mode"))));
@@ -16767,7 +16768,16 @@ ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
forced_dependency.oid = InvalidOid;
else
forced_dependency.oid = get_collation_oid(coll, false);
- strncpy(NameStr(forced_dependency.version), version, sizeof(NameData));
+
+ if (version)
+ strncpy(NameStr(forced_dependency.version), version, sizeof(NameData));
+ else
+ {
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ get_collation_version_for_oid(forced_dependency.oid,
+ &forced_dependency.version);
+ }
object.classId = RelationRelationId;
object.objectId = rel->rd_id;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4f53737e66..4d09f7bdac 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2567,6 +2567,15 @@ alter_table_cmd:
n->version = "";
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> DEPENDS ON ANY COLLATION UNKNOWN VERSION */
+ | DEPENDS ON COLLATION any_name CURRENT_P VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = $4;
+ n->version = NULL;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index df26826993..e869f3fc47 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -44,6 +44,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -792,6 +793,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1643,7 +1658,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "DEPENDS ON COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1689,6 +1704,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> DEPENDS ON COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS", "ON", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> DEPENDS ON COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS", "ON", "COLLATION", MatchAny))
+ COMPLETE_WITH("CURRENT VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4e559fea09..c16d633ed8 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,10 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
- char *version; /* version reference for collation dependency */
+ char *version; /* version reference for collation dependency.
+ * An empty string is used to represent an
+ * unknown version, and a NULL pointer is used
+ * to represent the current version */
RoleSpec *newowner;
Node *def; /* definition of new column, index,
* constraint, or parent table */
--
2.20.1
Re: Collation versioning
On Thu, Dec 12, 2019 at 2:45 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
Hearing no objection in [1], attached v5 with following changes:
- fix the spurious warnings by doing the version check in
get_relation_info and vacuum_open_relation, and add a flag in
RelationData to mark the check as already being done
- change the IsCatalogRelation() check to IsSystemRelation() to also
ignore toast indexes, as those can also be safely ignored too
- add a new ALTER INDEX idxname DEPENDS ON COLLATION collname CURRENT
VERSION to let users remove the warnings for safe library upgrade.
Documentation and tab completion addedIf I'm not mistaken, all issues I was aware of are now fixed.
Thanks! This is some great progress and I'm feeling positive about
getting this into PostgreSQL 13. I haven't (re)reviewed the code yet,
but I played with it a bit and have some more feedback.
There are some missing semi-colons on the ALTER INDEX statements in
pg_dump.c that make the pg_upgrade test fail (at least, if LC_ALL is
set).
We create duplicate pg_depend records:
postgres=# create table t (x text);
CREATE TABLE
postgres=# create index on t(x) where x > 'hello';
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_x_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
1259 | 16424 | 0 | 3456 | 100 | 0 |
0:34.0 | n
1259 | 16424 | 0 | 3456 | 100 | 0 |
0:34.0 | n
(2 rows)
I wondered if that was harmless, but for one thing it causes duplicate warnings:
postgres=# update pg_depend set refobjversion = 'BANANA' where
refobjversion = '0:34.0';
UPDATE 2
[new session]
postgres=# select count(*) from t;
WARNING: index "t_x_idx" depends on collation "default" version
"BANANA", but the current version is "0:34.0"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
WARNING: index "t_x_idx" depends on collation "default" version
"BANANA", but the current version is "0:34.0"
DETAIL: The index may be corrupted due to changes in sort order.
HINT: REINDEX to avoid the risk of corruption.
Here's another way to get a duplicate, and in this example you also
get an unnecessary dependency on 100 "default" for this index:
postgres=# create index on t(x collate "fr_FR") where x > 'helicopter'
collate "fr_FR";
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_x_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
1259 | 16460 | 0 | 3456 | 12603 | 0 |
0:34.0 | n
1259 | 16460 | 0 | 3456 | 12603 | 0 |
0:34.0 | n
1259 | 16460 | 0 | 3456 | 100 | 0 |
0:34.0 | n
(3 rows)
Or... maybe 100 should be there, by simple analysis of the x in the
WHERE clause, but it's the same if you write x collate "fr_FR" >
'helicopter' collate "fr_FR", and in that case there are no
expressions of collation "default" anywhere.
The indirection through composite types works nicely:
postgres=# create type foo_t as (en text collate "en_CA", fr text
collate "fr_CA");
CREATE TYPE
postgres=# create table t (foo foo_t);
CREATE TABLE
postgres=# create index on t(foo);
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_foo_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
1259 | 16444 | 0 | 3456 | 12554 | 0 |
0:34.0 | n
1259 | 16444 | 0 | 3456 | 12597 | 0 |
0:34.0 | n
(2 rows)
... but again it shows the extra and technically unnecessary
dependencies (only 12603 "fr_FR" is really needed):
postgres=# create index on t(((foo).fr collate "fr_FR"));
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_fr_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
1259 | 16445 | 0 | 3456 | 12603 | 0 |
0:34.0 | n
1259 | 16445 | 0 | 3456 | 12597 | 0 |
0:34.0 | n
1259 | 16445 | 0 | 3456 | 12554 | 0 |
0:34.0 | n
(3 rows)
I check that nested types are examined recursively, as appropriate. I
also tested domains, arrays, arrays of domains, expressions extracting
an element from an array of a domain with an explicit collation, and
the only problem I could find was more ways to get duplicates. Hmm...
what else is there that can contain a collatable type...? Ranges!
postgres=# create type myrange as range (subtype = text);
CREATE TYPE
postgres=# drop table t;
DROP TABLE
postgres=# create table t (x myrange);
CREATE TABLE
postgres=# create index on t(x);
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_x_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
... or perhaps, more realistically, a GIST index might actually be
useful for range queries, and we're not capturing the dependency:
postgres=# create index t_x_idx on t using gist (x);
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_x_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
The new syntax "ALTER INDEX i_name DEPENDS ON ANY COLLATION UNKNOWN
VERSION" doesn't sound good to me, it's not "ANY" collation, it's a
specific set of collations that we aren't listing. "ALTER INDEX
i_name DEPENDS ON COLLATION * VERSION UNKNOWN", hrmph, no that's
terrible... I'm not sure what would be better.
I'm not sure if I like the idea of VACUUM reporting warnings or not. Hmm.
To state more explicitly what's happening here, we're searching the
expression trees for subexpresions that have a collation as part of
their static type. We don't know which functions or operators are
actually affected by the collation, though. For example, if an
expression says "x IS NOT NULL" and x happens to be a subexpression of
a type with a particular collation, we don't now that this
expression's value can't possibly be affected by the collation version
changing. So, the system will nag you to rebuild an index just
because you mentioned it, even though the index can't be corrupted.
To do better than that, I suppose we'd need declarations in the
catalog to say which functions/operators are collation sensitive.
Then, as a special case, there is the collation of the actual indexed
value, because that will implicitly be used as input to the btree ops
that would be collation sensitive. That's just a thought experiment:
it seems like massive overkill to try to catalog collation sensitivity
for a rather limited benefit, and I'm happy with the way you have it.
More soon.
Re: Collation versioning
On Thu, Dec 12, 2019 at 5:00 PM Thomas Munro <thomas.munro@gmail.com> wrote:
Then, as a special case, there is the collation of the actual indexed
value, because that will implicitly be used as input to the btree ops
that would be collation sensitive. [...]
Erm, but I shouldn't have to reindex my hash indexes (at least not
until someone invents collation-based equality and therefore
necessarily also collation-based hashing). How can we exclude that?
amcanorder seems somehow right but also wrong.
Re: Collation versioning
Thomas Munro <thomas.munro@gmail.com> writes:
Erm, but I shouldn't have to reindex my hash indexes (at least not
until someone invents collation-based equality and therefore
necessarily also collation-based hashing). How can we exclude that?
Um, we already invented that with nondeterministic collations, no?
regards, tom lane
Re: Collation versioning
On Thu, Dec 12, 2019 at 6:32 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Thomas Munro <thomas.munro@gmail.com> writes:
Erm, but I shouldn't have to reindex my hash indexes (at least not
until someone invents collation-based equality and therefore
necessarily also collation-based hashing). How can we exclude that?Um, we already invented that with nondeterministic collations, no?
Urghlgh, right, thanks, somehow I missed/forgot that that stuff
already works for hashing (neat). So we do need to track collation
version dependencies for hash indexes, but only for non-deterministic
collations. I wonder how best to code that.
Re: Collation versioning
Hello Thomas,
Thanks for looking at it!
On Thu, Dec 12, 2019 at 5:01 AM Thomas Munro <thomas.munro@gmail.com> wrote:
We create duplicate pg_depend records:
[...]
I wondered if that was harmless
That's the assumed behavior of recordMultipleDependencies:
/*
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
We could add a check to skip duplicates for the "track_version ==
true" path, or switch to flags if we want to also skip duplicates in
other cases, but it'll make recordMultipleDependencies a little bit
more specialised.
but for one thing it causes duplicate warnings:
Yes, that should be avoided.
Here's another way to get a duplicate, and in this example you also
get an unnecessary dependency on 100 "default" for this index:postgres=# create index on t(x collate "fr_FR") where x > 'helicopter'
collate "fr_FR";
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_x_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
1259 | 16460 | 0 | 3456 | 12603 | 0 |
0:34.0 | n
1259 | 16460 | 0 | 3456 | 12603 | 0 |
0:34.0 | n
1259 | 16460 | 0 | 3456 | 100 | 0 |
0:34.0 | n
(3 rows)Or... maybe 100 should be there, by simple analysis of the x in the
WHERE clause, but it's the same if you write x collate "fr_FR" >
'helicopter' collate "fr_FR", and in that case there are no
expressions of collation "default" anywhere.
Ah good point. That's because expression_tree_walker() will dig into
CollateExpr->args and eventually reach the underlying Var. I don't
see an easy way to avoid that while still properly recording the
required dependency for an even more realistic index such as
CREATE INDEX ON t(x COLLATE "fr_FR") WHERE x > ((x COLLATE "en_US" >
'helicopter' COLLATE "en_US")::text) collate "fr_FR";
and for instance not for
CREATE INDEX ON t(x COLLATE "fr_FR") WHERE x > ((x COLLATE "en_US" ||
'helicopter' COLLATE "en_US")) collate "fr_FR";
The indirection through composite types works nicely:
postgres=# create type foo_t as (en text collate "en_CA", fr text
collate "fr_CA");
CREATE TYPE
postgres=# create table t (foo foo_t);
CREATE TABLE
postgres=# create index on t(foo);
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_foo_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
1259 | 16444 | 0 | 3456 | 12554 | 0 |
0:34.0 | n
1259 | 16444 | 0 | 3456 | 12597 | 0 |
0:34.0 | n
(2 rows)... but again it shows the extra and technically unnecessary
dependencies (only 12603 "fr_FR" is really needed):postgres=# create index on t(((foo).fr collate "fr_FR"));
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_fr_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
1259 | 16445 | 0 | 3456 | 12603 | 0 |
0:34.0 | n
1259 | 16445 | 0 | 3456 | 12597 | 0 |
0:34.0 | n
1259 | 16445 | 0 | 3456 | 12554 | 0 |
0:34.0 | n
(3 rows)
Yes :(
I check that nested types are examined recursively, as appropriate. I
also tested domains, arrays, arrays of domains, expressions extracting
an element from an array of a domain with an explicit collation, and
the only problem I could find was more ways to get duplicates. Hmm...
what else is there that can contain a collatable type...? Ranges!postgres=# create type myrange as range (subtype = text);
CREATE TYPE
postgres=# drop table t;
DROP TABLE
postgres=# create table t (x myrange);
CREATE TABLE
postgres=# create index on t(x);
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_x_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)... or perhaps, more realistically, a GIST index might actually be
useful for range queries, and we're not capturing the dependency:postgres=# create index t_x_idx on t using gist (x);
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_x_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
Good catch :) I fixed it locally and checked that a gist index on a
range with a subtype being a composite type does record the required
dependencies.
The new syntax "ALTER INDEX i_name DEPENDS ON ANY COLLATION UNKNOWN
VERSION" doesn't sound good to me, it's not "ANY" collation, it's a
specific set of collations that we aren't listing. "ALTER INDEX
i_name DEPENDS ON COLLATION * VERSION UNKNOWN", hrmph, no that's
terrible... I'm not sure what would be better.
Mmm, indeed. With a 3rd round in the existing keyword, how about
"DEPENDS ON [ ANY ] REFERENCING COLLATION"? The ANY is mostly to
avoid the need for plural.
I'm not sure if I like the idea of VACUUM reporting warnings or not. Hmm.
Even if I add this in a IsAutoVacuumWorkerProcess?
To state more explicitly what's happening here, we're searching the
expression trees for subexpresions that have a collation as part of
their static type. We don't know which functions or operators are
actually affected by the collation, though. For example, if an
expression says "x IS NOT NULL" and x happens to be a subexpression of
a type with a particular collation, we don't now that this
expression's value can't possibly be affected by the collation version
changing. So, the system will nag you to rebuild an index just
because you mentioned it, even though the index can't be corrupted.
To do better than that, I suppose we'd need declarations in the
catalog to say which functions/operators are collation sensitive.
Wouldn't that still be a problem for an absurd expression like
WHERE length((val collate "en_US" > 'uh' collate "en_US")::text) > 0
And since we would still have to record a dependency on the collation
in such case, we would need to have another magic value to distinguish
"unknown" from "cannot cause corruption" collation version.
Re: Collation versioning
On Thu, Dec 12, 2019 at 3:36 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
Hello Thomas,
Thanks for looking at it!
On Thu, Dec 12, 2019 at 5:01 AM Thomas Munro <thomas.munro@gmail.com> wrote:
We create duplicate pg_depend records:
[...]
I wondered if that was harmless
That's the assumed behavior of recordMultipleDependencies:
/*
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/We could add a check to skip duplicates for the "track_version ==
true" path, or switch to flags if we want to also skip duplicates in
other cases, but it'll make recordMultipleDependencies a little bit
more specialised.but for one thing it causes duplicate warnings:
Yes, that should be avoided.
Here's another way to get a duplicate, and in this example you also
get an unnecessary dependency on 100 "default" for this index:postgres=# create index on t(x collate "fr_FR") where x > 'helicopter'
collate "fr_FR";
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_x_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
1259 | 16460 | 0 | 3456 | 12603 | 0 |
0:34.0 | n
1259 | 16460 | 0 | 3456 | 12603 | 0 |
0:34.0 | n
1259 | 16460 | 0 | 3456 | 100 | 0 |
0:34.0 | n
(3 rows)Or... maybe 100 should be there, by simple analysis of the x in the
WHERE clause, but it's the same if you write x collate "fr_FR" >
'helicopter' collate "fr_FR", and in that case there are no
expressions of collation "default" anywhere.Ah good point. That's because expression_tree_walker() will dig into
CollateExpr->args and eventually reach the underlying Var. I don't
see an easy way to avoid that while still properly recording the
required dependency for an even more realistic index such asCREATE INDEX ON t(x COLLATE "fr_FR") WHERE x > ((x COLLATE "en_US" >
'helicopter' COLLATE "en_US")::text) collate "fr_FR";and for instance not for
CREATE INDEX ON t(x COLLATE "fr_FR") WHERE x > ((x COLLATE "en_US" ||
'helicopter' COLLATE "en_US")) collate "fr_FR";The indirection through composite types works nicely:
postgres=# create type foo_t as (en text collate "en_CA", fr text
collate "fr_CA");
CREATE TYPE
postgres=# create table t (foo foo_t);
CREATE TABLE
postgres=# create index on t(foo);
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_foo_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
1259 | 16444 | 0 | 3456 | 12554 | 0 |
0:34.0 | n
1259 | 16444 | 0 | 3456 | 12597 | 0 |
0:34.0 | n
(2 rows)... but again it shows the extra and technically unnecessary
dependencies (only 12603 "fr_FR" is really needed):postgres=# create index on t(((foo).fr collate "fr_FR"));
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_fr_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
1259 | 16445 | 0 | 3456 | 12603 | 0 |
0:34.0 | n
1259 | 16445 | 0 | 3456 | 12597 | 0 |
0:34.0 | n
1259 | 16445 | 0 | 3456 | 12554 | 0 |
0:34.0 | n
(3 rows)Yes :(
I check that nested types are examined recursively, as appropriate. I
also tested domains, arrays, arrays of domains, expressions extracting
an element from an array of a domain with an explicit collation, and
the only problem I could find was more ways to get duplicates. Hmm...
what else is there that can contain a collatable type...? Ranges!postgres=# create type myrange as range (subtype = text);
CREATE TYPE
postgres=# drop table t;
DROP TABLE
postgres=# create table t (x myrange);
CREATE TABLE
postgres=# create index on t(x);
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_x_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)... or perhaps, more realistically, a GIST index might actually be
useful for range queries, and we're not capturing the dependency:postgres=# create index t_x_idx on t using gist (x);
CREATE INDEX
postgres=# select * from pg_depend where objid = 't_x_idx'::regclass
and refobjversion != '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
refobjversion | deptype
---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)Good catch :) I fixed it locally and checked that a gist index on a
range with a subtype being a composite type does record the required
dependencies.The new syntax "ALTER INDEX i_name DEPENDS ON ANY COLLATION UNKNOWN
VERSION" doesn't sound good to me, it's not "ANY" collation, it's a
specific set of collations that we aren't listing. "ALTER INDEX
i_name DEPENDS ON COLLATION * VERSION UNKNOWN", hrmph, no that's
terrible... I'm not sure what would be better.Mmm, indeed. With a 3rd round in the existing keyword, how about
"DEPENDS ON [ ANY ] REFERENCING COLLATION"? The ANY is mostly to
avoid the need for plural.I'm not sure if I like the idea of VACUUM reporting warnings or not. Hmm.
Even if I add this in a IsAutoVacuumWorkerProcess?
To state more explicitly what's happening here, we're searching the
expression trees for subexpresions that have a collation as part of
their static type. We don't know which functions or operators are
actually affected by the collation, though. For example, if an
expression says "x IS NOT NULL" and x happens to be a subexpression of
a type with a particular collation, we don't now that this
expression's value can't possibly be affected by the collation version
changing. So, the system will nag you to rebuild an index just
because you mentioned it, even though the index can't be corrupted.
To do better than that, I suppose we'd need declarations in the
catalog to say which functions/operators are collation sensitive.Wouldn't that still be a problem for an absurd expression like
WHERE length((val collate "en_US" > 'uh' collate "en_US")::text) > 0
And since we would still have to record a dependency on the collation
in such case, we would need to have another magic value to distinguish
"unknown" from "cannot cause corruption" collation version.
PFA rebased v6, with the following changes:
- collation for range types is handled
- duplicated dependencies aren't recorded anymore
- relation column underlying type's collation isn't recorded if it's
used in an index expression and is directly under a CollateExpr node.
This should be safe as the required dependency is already recorded for
the relation's column, and should avoid most of the false positive
warning. With this modification, the following use case:
CREATE TABLE test (id integer, val text collate "en-x-icu"); CREATE
INDEX ON test ( val collate "ga-x-icu" ) WHERE ((val collate
"fr-x-icu") collate "es-x-icu") is not null;
stores the following dependencies for collations:
SELECT refobjid::regcollation, objid::regclass FROM pg_depend WHERE
objid::regclass::text ILIKE 'test%' AND refclassid = 3456;
refobjid | objid
------------+--------------
"en-x-icu" | test
"ga-x-icu" | test_val_idx
"fr-x-icu" | test_val_idx
"es-x-icu" | test_val_idx
(4 rows)
Unless we add a distinct catalog for version dependencies or a
specific magic value, we have to record the "fr-x-icu" and "ex-x-icu"
dependencies.
- for hash indexes, deterministic collations aren't track if it's a
simple key column. I added a new "non_deterministic_only" parameter
to GetTypeCollations() and modified the loop over key attributes in
index_create(). With the previous example modified to be a hash
index, "ga-x-icu" wouldn't be recorded.
Attachments:
0005-Preserve-index-dependencies-on-collation-during-pg_u-v6.patchapplication/octet-stream; name=0005-Preserve-index-dependencies-on-collation-during-pg_u-v6.patchDownload
From 9822fd2440dae8203bb3386bc27dd925300ab464 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/6] Preserve index dependencies on collation during
pg_upgrade
Two new commands are added.
ALTER INDEX i_name DEPENDS ON COLLATION c_name VERSION v_name
to force a dependency on a collation specific version, and
ALTER INDEX i_name DEPENDS ON ANY COLLATION UNKOWN VERSION
to specify that the version is unknown for all collations on a specific index.
Both commands are only allowed in binary upgrade mode. Also teach pg_dump to
emit such commands for all indexes, including indexes created for constraints,
when run with --binary-upgrade flag.
When pg_upgrade is run against an older version, collation versions are not
known and pg_dump will by default emit an alter index command to mark all
collation versions as unkown. However, it's possible that pg_upgrade is run
without upgrading the underlying collation libraries, so a new option
--collation-binary-compatible is added to avoid this behavior. This will
result in running pg_dump with a new --unknown-collations-binary-compatible
option, that can only be used in binary upgrade mode, to prevent pg_dump from
emitting the alter index commands if the dependent collation version is
unknown. Note that if the collation version is known, this flag won't change
the behavior and the previous collation version will be preserved.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 16 +++
src/backend/catalog/index.c | 16 ++-
src/backend/commands/tablecmds.c | 75 +++++++++++++
src/backend/nodes/copyfuncs.c | 2 +
src/backend/parser/gram.y | 18 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 179 +++++++++++++++++++++++++++++--
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 ++
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/nodes/parsenodes.h | 5 +-
12 files changed, 314 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index d4da566d76..40e8a5e919 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,22 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ When upgrading from a PostgreSQL major version 12 or older, all indexes
+ will be marked as depending on an unknown collation version, as such
+ versions weren't tracked. As a result, numerous warning messages will
+ be emitted as it can be a sign of a corrupted index. If you're not
+ upgrading the collation libraries, and if you're absolutly certain that
+ all existing indexes are compatible with the current collation
+ libraries, you can use this flag to change this behavior and mark all
+ indexes as depending on current collation libraries version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index ce9e383d13..8a44cdbf90 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1277,11 +1277,24 @@ index_check_collation_version(const ObjectAddress *otherObject,
/* Compare with the current version. */
get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+
if (strncmp(NameStr(*version),
NameStr(current_version),
sizeof(NameData)) != 0)
{
- ereport(WARNING,
+ if (strncmp(NameStr(*version), "", sizeof(NameData)) == 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else
+ {
+ ereport(WARNING,
(errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
get_rel_name(relid),
get_collation_name(otherObject->objectId),
@@ -1289,6 +1302,7 @@ index_check_collation_version(const ObjectAddress *otherObject,
NameStr(current_version)),
errdetail("The index may be corrupted due to changes in sort order."),
errhint("REINDEX to avoid the risk of corruption.")));
+ }
}
return NULL;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index daa80ec4aa..67d1fd5732 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -201,6 +201,13 @@ typedef struct NewColumnValue
ExprState *exprstate; /* execution state */
} NewColumnValue;
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ NameData version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/*
* Error-reporting support for RemoveRelations
*/
@@ -527,6 +534,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecDependsOnCollationVersion(Relation rel, List *coll,
+ char *version);
/* ----------------------------------------------------------------
@@ -3804,6 +3813,11 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ /* Only used in binary upgrade mode */
+ case AT_DependsOnCollationVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -3957,6 +3971,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
+ * [UNKNOWN VERSION | VERSION ...] */
+ if (!IsBinaryUpgrade)
+ ereport(ERROR,
+ (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ (errmsg("command can only be called when server is in binary upgrade mode"))));
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode);
@@ -4481,6 +4505,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_DependsOnCollationVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecDependsOnCollationVersion(rel, cmd->object, cmd->version);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -16703,3 +16732,49 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+static NameData *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return &forced_dependency->version;
+}
+
+static void
+ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ if (coll == NIL)
+ forced_dependency.oid = InvalidOid;
+ else
+ forced_dependency.oid = get_collation_oid(coll, false);
+ strncpy(NameStr(forced_dependency.version), version, sizeof(NameData));
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ebba5d4fa9..f63792500a 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3168,7 +3168,9 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
+ COPY_STRING_FIELD(version);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
COPY_SCALAR_FIELD(behavior);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8ba21cefcf..4f53737e66 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2549,6 +2549,24 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> DEPENDS ON COLLATION ... VERSION ... */
+ | DEPENDS ON COLLATION any_name VERSION_P Sconst
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = $4;
+ n->version = $6;
+ $$ = (Node *)n;
+ }
+ /* ALTER INDEX <name> DEPENDS ON ANY COLLATION UNKNOWN VERSION */
+ | DEPENDS ON ANY COLLATION UNKNOWN VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = NIL;
+ n->version = "";
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index cd40a55845..7ed44355e9 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -288,6 +289,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -388,6 +391,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -708,6 +712,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6841,7 +6849,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6877,7 +6887,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid::pg_catalog.regcollation ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion != '') AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(refobjversion ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion != '') AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6902,7 +6967,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6941,7 +7008,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6976,7 +7045,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7007,7 +7078,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7041,7 +7114,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7081,6 +7156,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7106,6 +7183,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16374,10 +16453,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16437,6 +16517,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16466,6 +16550,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18423,6 +18522,64 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of ALTER INDEX ... DEPENDS ON COLLATION ... VERSION ....
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, "") == 0);
+
+ appendPQExpBuffer(buffer, "ALTER INDEX %s",
+ fmtQualifiedDumpable(indxinfo));
+ appendPQExpBuffer(buffer, " DEPENDS ON ANY COLLATION UNKNOWN VERSION;\n");
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ appendPQExpBuffer(buffer, "ALTER INDEX %s",
+ fmtQualifiedDumpable(indxinfo));
+ appendPQExpBuffer(buffer, " DEPENDS ON COLLATION %s VERSION ",
+ inddependoidsarray[i]);
+ appendStringLiteral(buffer, inddependversionsarray[i], enc, true);
+ appendPQExpBuffer(buffer, ";\n");
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 7b2c1524a5..6f67d195dd 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index bdb5006fa6..bc9480d8cf 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 14351e8028..ffa3394cc5 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 729f86aa32..202a8b67bd 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -294,6 +294,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 6b4e188e81..4e559fea09 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1824,7 +1824,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_DependsOnCollationVersion /* DEPENDS ON COLLATION ... VERSION ... */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1840,8 +1841,10 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
+ char *version; /* version reference for collation dependency */
RoleSpec *newowner;
Node *def; /* definition of new column, index,
* constraint, or parent table */
--
2.20.1
0003-Track-collation-versions-for-indexes-v6.patchapplication/octet-stream; name=0003-Track-collation-versions-for-indexes-v6.patchDownload
From 825f7740061f3409217581b7bb2add457d65ad45 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 3/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 171 ++++++++++++++++++---
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 126 +++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 130 ++++++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++++
src/backend/commands/vacuum.c | 31 ++++
src/backend/optimizer/util/plancat.c | 9 ++
src/backend/utils/adt/pg_locale.c | 39 +++++
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 21 ++-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 1 +
src/include/utils/rel.h | 1 +
src/test/regress/expected/create_index.out | 8 +-
16 files changed, 561 insertions(+), 60 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 0bb1504b20..84e2769a3c 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -137,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +440,63 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ NameData *new_version;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ new_version = callback(&otherObject, &foundDep->refobjversion,
+ userdata);
+ if (new_version)
+ {
+ /* Make a modifyable copy. */
+ tup = heap_copytuple(tup);
+ foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ foundDep->refobjversion = *new_version;
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1650,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1666,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1696,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1761,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1783,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1741,8 +1814,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1775,6 +1853,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
else if (rte->rtekind == RTE_JOIN)
{
@@ -1808,11 +1926,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1902,7 +2022,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1990,7 +2111,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2021,7 +2143,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2034,7 +2157,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2047,7 +2171,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2136,7 +2261,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2254,7 +2380,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2276,7 +2404,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2672,8 +2802,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 8404904710..f9d9ef96c8 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2273,7 +2273,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2283,7 +2283,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3527,7 +3527,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e9955707fa..ce9e383d13 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -725,6 +726,7 @@ index_create(Relation heapRelation,
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
+ List *collations = NIL;
/* constraint flags can only be set when a constraint is requested */
Assert((constr_flags == 0) ||
@@ -1116,21 +1118,44 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /*
+ * Get required distinct dependencies on collations for all index keys.
+ * Collations of directly referenced column in hash indexes can be
+ * skipped is they're deterministic.
+ */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ if ((indexInfo->ii_Am != HASH_AM_OID) ||
+ !get_collation_isdeterministic(colloid))
+ collations = list_append_unique_oid(collations, colloid);
+ }
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ collations = list_concat_unique_oid(collations,
+ GetTypeCollations(att->atttypid,
+ (indexInfo->ii_Am == HASH_AM_OID)));
}
}
+ if (collations)
+ {
+ recordDependencyOnCollations(&myself, collations);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1144,21 +1169,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1230,6 +1263,76 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static NameData *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ NameData current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ get_collation_version_for_oid(otherObject->objectId, ¤t_version);
+ if (strncmp(NameStr(*version),
+ NameStr(current_version),
+ sizeof(NameData)) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ NameStr(*version),
+ NameStr(current_version)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static NameData *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata)
+{
+ NameData *current_version = (NameData *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ get_collation_version_for_oid(otherObject->objectId, current_version);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3597,6 +3700,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 56568b0105..a271eb2e2d 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -361,7 +361,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f1dc1143a7..c69d6c6d36 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -26,9 +27,12 @@
#include "miscadmin.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,22 +48,29 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, true);
+ }
}
/*
@@ -69,9 +80,9 @@ recordDependencyOnVersion(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +90,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ NameData version;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -99,12 +111,48 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ version.data[0] = '\0';
+
+ if (track_version)
+ {
+ /* Only dependency on a collation needs to be tracked */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;;
+ get_collation_version_for_oid(referenced->objectId, &version);
+ }
+ }
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -117,7 +165,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
- values[Anum_pg_depend_refobjversion - 1] = version ? NameGetDatum(version) : CStringGetDatum("");
+ values[Anum_pg_depend_refobjversion - 1] = NameGetDatum(&version);
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
@@ -555,6 +603,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index a8c1de511f..ec224580d6 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index da1da23400..d0841ea657 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -579,6 +581,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 5e889d1861..7e10568953 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index b7a4a1421e..59b46fcf52 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -1501,6 +1503,43 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ */
+void
+get_collation_version_for_oid(Oid oid, NameData *output)
+{
+ HeapTuple tp;
+ const char *version;
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ memset(output, 0, sizeof(NameData));
+ if (version)
+ strncpy(NameStr(*output), version, sizeof(NameData));
+ ReleaseSysCache(tp);
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 50f8912c13..4fed56a17c 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5637,6 +5638,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b5d3276e77..eb34616635 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,28 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef NameData *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const NameData *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 27d9e537d3..f8b7a5ed54 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 0c273a0449..17f89a3f71 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index b4b3aa5843..023ef693e1 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern void get_collation_version_for_oid(Oid collid, NameData *output);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 31d8a1a10e..c6ed5d3e5a 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -62,6 +62,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 645ae2cf34..46c0820eb3 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2003,15 +2003,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2031,15 +2033,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
--
2.20.1
0002-Add-pg_depend.refobjversion-v6.patchapplication/octet-stream; name=0002-Add-pg_depend.refobjversion-v6.patchDownload
From 473b360079cb2bd5dfefa186a3a0d4c8e98ad565 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 20 +++++++++--
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
6 files changed, 56 insertions(+), 30 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index d07bb4496e..0bb1504b20 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2669,7 +2672,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index a060c25d2e..f1dc1143a7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -101,9 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ values[Anum_pg_depend_refobjversion - 1] = version ? NameGetDatum(version) : CStringGetDatum("");
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 1f6d8939be..0909723aaf 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1562,55 +1562,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0,'', 'p' "
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ff50d594f6..b5d3276e77 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index f786445fb2..e30896178a 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -55,6 +55,7 @@ CATALOG(pg_depend,2608,DependRelationId)
Oid refclassid; /* OID of table containing object */
Oid refobjid; /* OID of object itself */
int32 refobjsubid; /* column number, or 0 if not used */
+ NameData refobjversion; /* version tracking, or empty if not used */
/*
* Precise semantics of the relationship are specified by the deptype
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..44d5e756a3 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | refobjversion | deptype
+---------+-------+----------+------------+----------+-------------+---------------+---------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
0006-Add-a-new-ALTER-INDEX-name-DEPENDS-ON-COLLATION-name-v6.patchapplication/octet-stream; name=0006-Add-a-new-ALTER-INDEX-name-DEPENDS-ON-COLLATION-name-v6.patchDownload
From 349bdf1827e88781501dac424addd82d7fb081ae Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH 6/6] Add a new ALTER INDEX name DEPENDS ON COLLATION name
CURRENT VERSION
This command allows privileged users to specifify that the currently installed
collation library version, for a specific collation, is binary compatible with
the one that was installed when the specified index was built for the last
time.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 13 +++++++++++++
src/backend/commands/tablecmds.c | 14 ++++++++++++--
src/backend/parser/gram.y | 9 +++++++++
src/bin/psql/tab-complete.c | 26 +++++++++++++++++++++++++-
src/include/nodes/parsenodes.h | 5 ++++-
5 files changed, 63 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..f4a4333ab1 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON COLLATION <replaceable class="parameter">collation_name</replaceable> CURRENT VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,18 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>DEPENDS ON COLLATION</literal></term>
+ <listitem>
+ <para>
+ This form update the index existing dependency on a specific collation,
+ to specificy the the currently installed collation version is compatible
+ with the version used the last time the index was built. Be aware that
+ an incorrect use of this form can hide a corruption on the index.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 67d1fd5732..5a7da64f95 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -92,6 +92,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -3973,7 +3974,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
break;
case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
* [UNKNOWN VERSION | VERSION ...] */
- if (!IsBinaryUpgrade)
+ if (!IsBinaryUpgrade && cmd->version)
ereport(ERROR,
(errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
(errmsg("command can only be called when server is in binary upgrade mode"))));
@@ -16767,7 +16768,16 @@ ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
forced_dependency.oid = InvalidOid;
else
forced_dependency.oid = get_collation_oid(coll, false);
- strncpy(NameStr(forced_dependency.version), version, sizeof(NameData));
+
+ if (version)
+ strncpy(NameStr(forced_dependency.version), version, sizeof(NameData));
+ else
+ {
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ get_collation_version_for_oid(forced_dependency.oid,
+ &forced_dependency.version);
+ }
object.classId = RelationRelationId;
object.objectId = rel->rd_id;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4f53737e66..4d09f7bdac 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2567,6 +2567,15 @@ alter_table_cmd:
n->version = "";
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> DEPENDS ON ANY COLLATION UNKNOWN VERSION */
+ | DEPENDS ON COLLATION any_name CURRENT_P VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = $4;
+ n->version = NULL;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index df26826993..e869f3fc47 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -44,6 +44,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -792,6 +793,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1643,7 +1658,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "DEPENDS ON COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1689,6 +1704,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> DEPENDS ON COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS", "ON", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> DEPENDS ON COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS", "ON", "COLLATION", MatchAny))
+ COMPLETE_WITH("CURRENT VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4e559fea09..c16d633ed8 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,10 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
- char *version; /* version reference for collation dependency */
+ char *version; /* version reference for collation dependency.
+ * An empty string is used to represent an
+ * unknown version, and a NULL pointer is used
+ * to represent the current version */
RoleSpec *newowner;
Node *def; /* definition of new column, index,
* constraint, or parent table */
--
2.20.1
0001-Remove-pg_collation.collversion-v6.patchapplication/octet-stream; name=0001-Remove-pg_collation.collversion-v6.patchDownload
From 37ad7d923b0d46e5d2aeda0aaa70193cfc150d00 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/6] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
18 files changed, 11 insertions(+), 290 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 55694c4368..21adb1640a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2127,17 +2127,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 57a1539506..22af58b262 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21655,10 +21655,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index dd99d53547..e67db0d291 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 919e092483..4f1442df03 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index a74b56bb59..ebba5d4fa9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3177,16 +3177,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5166,9 +5156,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 2fcd4a3467..aa786b563e 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1101,14 +1101,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3262,9 +3254,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c5086846de..8ba21cefcf 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -827,7 +827,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10298,21 +10297,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 3a03ca7e2f..5a67901e21 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1685,10 +1685,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2789,10 +2785,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3401,10 +3393,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index fcdbaae37b..b7a4a1421e 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 08658c8e86..cd40a55845 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13485,7 +13485,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13557,7 +13561,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index 367ce3607b..7f88a41ad4 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index d3366f361d..44c3b5fd85 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index cc5dfed0bf..1fd27d5fd3 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index e25f5d50b3..8090a5cb0c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ff626cbe61..6b4e188e81 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1850,17 +1850,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
0004-Implement-type-regcollation-v6.patchapplication/octet-stream; name=0004-Implement-type-regcollation-v6.patchDownload
From 3833f13588b5dfa407686ef08677b1d0391bdf9f Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH 4/6] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
4 files changed, 185 insertions(+)
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index 4b1decf81e..7ced3f400d 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index aabfa7af03..e4c7dec4d9 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ac8f64b219..aef574c46c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6636,6 +6636,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7422,6 +7431,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index d9b35af914..5d0b077e4a 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
Re: Collation versioning
On 2019-12-17 14:25, Julien Rouhaud wrote:
PFA rebased v6, with the following changes:
Some thoughts on this patch set.
I think we are all agreed on the general idea.
0001-0003 seem pretty much OK. Why is pg_depend.refobjversion of type
"name" whereas the previous pg_collation.collversion was type "text"?
Related to that, we previously used null to indicate an unknown
collation version, and now it's an empty string.
Also, this would limit collation versions to 63 characters. Perhaps not
a problem right now, but if someone wants to implement Thomas's previous
md5-the-file idea with sha256, we'll run out of space.
For 0005, if the new commands are only to be used in binary upgrades,
then they should be implemented as built-in functions instead of full
DDL commands. There is precedent for that.
The documentation snippet for this patch talks about upgrades from PG12
to later. But what about upgrades on platforms where we currently don't
have collation versioning but might introduce it later (FreeBSD,
Windows)? This needs to be generalized.
For 0006 ("Add a new ALTER INDEX name DEPENDS ON COLLATION name
CURRENT VERSION"), I find the syntax misleading. This command doesn't
(or shouldn't) add a new dependency, it only changes the version of an
existing dependency. The previously used syntax ALTER COLLATION /
REFRESH VERSION is a better vocabulary, I think.
I think this whole thing needs more tests. We are designing this
delicate mechanism but have no real tests for it. We at least need to
come up with a framework for how to test this automatically, so that we
can add more test cases over time as people will invariably report
issues when this hits the real world.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On Tue, Jan 28, 2020 at 1:06 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 2019-12-17 14:25, Julien Rouhaud wrote:
PFA rebased v6, with the following changes:
Some thoughts on this patch set.
Thanks for looking at it!
I think we are all agreed on the general idea.
0001-0003 seem pretty much OK. Why is pg_depend.refobjversion of type
"name" whereas the previous pg_collation.collversion was type "text"?
Related to that, we previously used null to indicate an unknown
collation version, and now it's an empty string.
That's what Thomas implemented when he started to work on it and I
simply kept it that way until now. I'm assuming that it was simply to
avoid wasting time on the varlena stuff while working on the
prototype, so barring any objections I'll change to nullable text
column in the next revision.
For 0005, if the new commands are only to be used in binary upgrades,
then they should be implemented as built-in functions instead of full
DDL commands. There is precedent for that.
Oh, I wasn't aware of that. I can definitely use built-in functions
instead, but some people previously argued that those command should
be available even in non binary upgrade and I'm not clear on whether
this is wanted or not. Do you have any thoughts on that?
The documentation snippet for this patch talks about upgrades from PG12
to later. But what about upgrades on platforms where we currently don't
have collation versioning but might introduce it later (FreeBSD,
Windows)? This needs to be generalized.
Good point, I'll try to improve that.
For 0006 ("Add a new ALTER INDEX name DEPENDS ON COLLATION name
CURRENT VERSION"), I find the syntax misleading. This command doesn't
(or shouldn't) add a new dependency, it only changes the version of an
existing dependency. The previously used syntax ALTER COLLATION /
REFRESH VERSION is a better vocabulary, I think.
I agree and also complained about that syntax too. I'm however
struggling on coming up with a syntax that makes it clear it's
refreshing the version of a collation the index already depends on.
E.g.:
ALTER INDEX name ALTER COLLATION name REFRESH VERSION
is still quite poor, but I don't have anything better. Do you have
some better suggestion or should I go with that?
I think this whole thing needs more tests. We are designing this
delicate mechanism but have no real tests for it. We at least need to
come up with a framework for how to test this automatically, so that we
can add more test cases over time as people will invariably report
issues when this hits the real world.
Indeed. I have some unlikely index test cases I'm for now using to
validate the behavior, but didn't start a real test infrastructure.
I'll also work on that for the next revision, although I'll need some
more thinking on how to properly test the pg_upgrade part.
Re: Collation versioning
On Wed, Jan 29, 2020 at 9:58 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Tue, Jan 28, 2020 at 1:06 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:0001-0003 seem pretty much OK. Why is pg_depend.refobjversion of type
"name" whereas the previous pg_collation.collversion was type "text"?
Related to that, we previously used null to indicate an unknown
collation version, and now it's an empty string.That's what Thomas implemented when he started to work on it and I
simply kept it that way until now. I'm assuming that it was simply to
avoid wasting time on the varlena stuff while working on the
prototype, so barring any objections I'll change to nullable text
column in the next revision.
Done
The documentation snippet for this patch talks about upgrades from PG12
to later. But what about upgrades on platforms where we currently don't
have collation versioning but might introduce it later (FreeBSD,
Windows)? This needs to be generalized.Good point, I'll try to improve that.
Done.
I think this whole thing needs more tests. We are designing this
delicate mechanism but have no real tests for it. We at least need to
come up with a framework for how to test this automatically, so that we
can add more test cases over time as people will invariably report
issues when this hits the real world.Indeed. I have some unlikely index test cases I'm for now using to
validate the behavior, but didn't start a real test infrastructure.
I'll also work on that for the next revision, although I'll need some
more thinking on how to properly test the pg_upgrade part.
So I added all tests I could think of to validate the correct behavior
of all the new stuff. Mostly:
- tests to make sure that we properly track a collation version for
various case of collation hidden somewhere in index definitions
- tests for pg_dump in binary upgrade mode to make sure that the
collation version (or the lack of known version) is correctly
preserved. I also modified pg_dump TAP tests to restore the binary
dump on an instance in binary mode, redump it and rerun the related
testsuite. While doing that I also realized that the previous support
for unknown version for partly broken, as it's possible to end up with
a database where only part of the collation versions are none. I
fixed it, adding a new "DEPEND ON COLLATION x UNKNOWN VERSION"
alternative for that case, with proper pg_dump support and required
tests
- new pg_dump TAP test mode with both --binary-upgrade and the new
--unknown-collations-binary-compatible switch, to make sure that we
don't dump the UNKNOWN VERSION orders when the
--collation-binary-compatible pg_upgrade option is used
- test for the ALTER INDEX name DEPENDS ON COLLATION name CURRENT VERSION
To avoid too many platform dependent behavior, I restricted the tests
for ICU collation only, with the required changes to ignore the tests
when postgres isn't compile with ICU support. One exception is a
couple of test to validate that we correctly add dependencies for
default collation, as we for now only support libc default collation.
It means that as written, the collate.icu.utf8 test will need an
alternative output for glibc platforms (which I didn't added yet, as
I'm sure there'll be other changes required, so let's avoid the pain
of maintaining it for now), as I've been testing the expected file
against macos.
Note that I didn't change any syntax (or switched to native functions
for the binary pg_dump) as it's still not clear to me what exactly
should be implemented.
Attachments:
0001-Remove-pg_collation.collversion-v7.patchapplication/octet-stream; name=0001-Remove-pg_collation.collversion-v7.patchDownload
From a1fd3c1f1ad15bc36a8da25978b728b51ef54a14 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/6] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
18 files changed, 11 insertions(+), 290 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index a10b66569b..f187a1af65 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index ceda48e0fc..44d82ca0b3 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21103,10 +21103,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 85f726ae06..493aa21a14 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 54ad62bb7f..bf4f793ba2 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3183,16 +3183,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5172,9 +5162,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 5b1ba143b1..4ec777a78c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3269,9 +3261,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 1b0edf5d3d..b6d6a0e239 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10321,21 +10320,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index bb85b5e52a..e57e05b9ee 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1798,10 +1798,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2948,10 +2944,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 25fb7e2ebf..597c1241f9 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ec3e2c63b0..9aa6496814 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13485,7 +13485,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13557,7 +13561,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index df7d1d498c..9a9e145b4c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index da0706add5..c96d027362 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
0003-Implement-type-regcollation-v7.patchapplication/octet-stream; name=0003-Implement-type-regcollation-v7.patchDownload
From 54374da19005e2fa1373693f05f0f87799281337 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH 3/6] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
4 files changed, 185 insertions(+)
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2228256907..b93ddeb720 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6663,6 +6663,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7449,6 +7458,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index fe2c4eabb4..012c17bb68 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
0002-Add-pg_depend.refobjversion-v7.patchapplication/octet-stream; name=0002-Add-pg_depend.refobjversion-v7.patchDownload
From d9b246584258ef79dddfb629f43675958a413106 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 28 ++++++++++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 3 ++
src/test/regress/expected/misc_sanity.out | 7 ++--
6 files changed, 66 insertions(+), 33 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index c4a4df25b8..78c31baa34 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2681,7 +2684,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f9af245eec..7fdbdf0ae8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 7f1534aebb..e72f85d8f0 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0cd6fcf027..77cf0612ed 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..41efb4a2c8 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
@@ -103,9 +103,10 @@ ORDER BY 1, 2;
pg_class | relacl | aclitem[]
pg_class | reloptions | text[]
pg_class | relpartbound | pg_node_tree
+ pg_depend | refobjversion | text
pg_index | indexprs | pg_node_tree
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
--
2.20.1
0005-Preserve-index-dependencies-on-collation-during-pg_u-v7.patchapplication/octet-stream; name=0005-Preserve-index-dependencies-on-collation-during-pg_u-v7.patchDownload
From c8db7e6b415dc85410f5e240a5ee575bbc373d36 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/6] Preserve index dependencies on collation during
pg_upgrade
Two new commands are added.
ALTER INDEX i_name DEPENDS ON COLLATION c_name VERSION v_name
to force a dependency on a collation specific version, and
ALTER INDEX i_name DEPENDS ON ANY COLLATION UNKOWN VERSION
to specify that the version is unknown for all collations on a specific index.
Both commands are only allowed in binary upgrade mode. Also teach pg_dump to
emit such commands for all indexes, including indexes created for constraints,
when run with --binary-upgrade flag.
When pg_upgrade is run against an older version, collation versions are not
known and pg_dump will by default emit an alter index command to mark all
collation versions as unkown. However, it's possible that pg_upgrade is run
without upgrading the underlying collation libraries, so a new option
--collation-binary-compatible is added to avoid this behavior. This will
result in running pg_dump with a new --unknown-collations-binary-compatible
option, that can only be used in binary upgrade mode, to prevent pg_dump from
emitting the alter index commands if the dependent collation version is
unknown. Note that if the collation version is known, this flag won't change
the behavior and the previous collation version will be preserved.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 +++
src/backend/commands/tablecmds.c | 83 +++++++++++
src/backend/nodes/copyfuncs.c | 2 +
src/backend/parser/gram.y | 27 ++++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 193 ++++++++++++++++++++++--
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 244 +++++++++++++++++++++++--------
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/nodes/parsenodes.h | 5 +-
src/test/perl/PostgresNode.pm | 6 +-
14 files changed, 524 insertions(+), 73 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index d4da566d76..6c55f3b55a 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ When upgrading from a PostgreSQL major version 12 or older, or from a
+ PostgreSQL major version that didn't have support for collation
+ versioning to a major version that now supports it, all indexes will be
+ marked as depending on an unknown collation version, as such versions
+ weren't tracked. As a result, numerous warning messages will be
+ emitted as it can be a sign of a corrupted index. If you're not
+ upgrading the collation libraries, and if you're absolutly certain that
+ all existing indexes are compatible with the current collation
+ libraries, you can use this flag to change this behavior and mark all
+ indexes as depending on current collation libraries version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f599393473..b3cf5641fd 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -207,6 +207,13 @@ typedef struct NewColumnValue
bool is_generated; /* is it a GENERATED expression? */
} NewColumnValue;
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/*
* Error-reporting support for RemoveRelations
*/
@@ -553,6 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecDependsOnCollationVersion(Relation rel, List *coll,
+ char *version);
/* ----------------------------------------------------------------
@@ -3871,6 +3880,11 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ /* Only used in binary upgrade mode */
+ case AT_DependsOnCollationVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4038,6 +4052,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
+ * [UNKNOWN VERSION | VERSION ...] */
+ if (!IsBinaryUpgrade)
+ ereport(ERROR,
+ (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ (errmsg("command can only be called when server is in binary upgrade mode"))));
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4604,6 +4628,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_DependsOnCollationVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecDependsOnCollationVersion(rel, cmd->object, cmd->version);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17263,3 +17292,57 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION DEPENDS ON ...
+ *
+ * A version has to be provided. If the caller wants to notify that the
+ * collation version to depend on is unknown, an empty string is passed.
+ */
+static void
+ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ if (coll == NIL)
+ forced_dependency.oid = InvalidOid;
+ else
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index bf4f793ba2..8b1e8e73c0 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3174,7 +3174,9 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
+ COPY_STRING_FIELD(version);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
COPY_SCALAR_FIELD(behavior);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index b6d6a0e239..1437123540 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2569,6 +2569,33 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> DEPENDS ON COLLATION ... VERSION ... */
+ | DEPENDS ON COLLATION any_name VERSION_P Sconst
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = $4;
+ n->version = $6;
+ $$ = (Node *)n;
+ }
+ /* ALTER INDEX <name> DEPENDS ON COLLATION ... UNKNOWN VERSION */
+ | DEPENDS ON COLLATION any_name UNKNOWN VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = $4;
+ n->version = "";
+ $$ = (Node *)n;
+ }
+ /* ALTER INDEX <name> DEPENDS ON ANY COLLATION UNKNOWN VERSION */
+ | DEPENDS ON ANY COLLATION UNKNOWN VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = NIL;
+ n->version = "";
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9aa6496814..5efb26d5df 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -288,6 +289,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -388,6 +391,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -708,6 +712,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6841,7 +6849,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6877,7 +6887,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid::pg_catalog.regcollation ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(refobjversion ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6902,7 +6967,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6941,7 +7008,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6976,7 +7045,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7007,7 +7078,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7041,7 +7114,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7081,6 +7156,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7106,6 +7183,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16374,10 +16453,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16437,6 +16517,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16466,6 +16550,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18423,6 +18522,78 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of ALTER INDEX ... DEPENDS ON COLLATION ... VERSION ....
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBuffer(buffer, "ALTER INDEX %s",
+ fmtQualifiedDumpable(indxinfo));
+ appendPQExpBuffer(buffer, " DEPENDS ON ANY COLLATION UNKNOWN VERSION;\n");
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ if (strcmp(inddependversionsarray[i], "") == 0)
+ {
+ if (!unknown_coll_compat)
+ {
+ appendPQExpBuffer(buffer, "ALTER INDEX %s",
+ fmtQualifiedDumpable(indxinfo));
+ appendPQExpBuffer(buffer, " DEPENDS ON COLLATION %s "
+ "UNKNOWN VERSION;\n",
+ inddependoidsarray[i]);
+ }
+ }
+ else
+ {
+ appendPQExpBuffer(buffer, "ALTER INDEX %s",
+ fmtQualifiedDumpable(indxinfo));
+ appendPQExpBuffer(buffer, " DEPENDS ON COLLATION %s VERSION ",
+ inddependoidsarray[i]);
+ appendStringLiteral(buffer, inddependversionsarray[i], enc, true);
+ appendPQExpBuffer(buffer, ";\n");
+ }
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 21004e5078..6d1df24080 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 4a9764c2d2..57172b57b3 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -914,9 +933,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1178,6 +1198,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1203,6 +1224,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1238,6 +1260,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1260,6 +1283,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1281,6 +1305,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1302,6 +1327,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1665,6 +1691,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1679,7 +1706,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2347,6 +2374,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2540,6 +2568,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2607,6 +2636,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2678,6 +2708,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3145,6 +3176,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3160,6 +3192,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3292,16 +3325,49 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ 'ALTER INDEX DEPENDS ON COLLATION VERSION' => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/ALTER INDEX dump_test.regress_coll_idx1 DEPENDS ON COLLATION "fr-x-icu" VERSION 'not_a_version'/,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ 'ALTER INDEX DEPENDS ON COLLATION UNKNOWN VERSION' => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/ALTER INDEX dump_test.regress_coll_no_ver_idx1 DEPENDS ON COLLATION "fr-x-icu" UNKNOWN VERSION/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3325,6 +3391,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3375,16 +3445,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3432,6 +3515,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3485,79 +3574,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 8d207b20ed..7daa251d4c 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index b156b516cc..af34dff920 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -294,6 +294,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index c96d027362..b36824ddfe 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_DependsOnCollationVersion /* DEPENDS ON COLLATION ... VERSION ... */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,8 +1861,10 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
+ char *version; /* version reference for collation dependency */
RoleSpec *newowner;
Node *def; /* definition of new column, index,
* constraint, or parent table */
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index bf095a7adb..d5c7866d2f 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
0004-Track-collation-versions-for-indexes-v7.patchapplication/octet-stream; name=0004-Track-collation-versions-for-indexes-v7.patchDownload
From 06d6549d8b0c4f9987ba518328e8ff886d50e23f Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 4/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 189 ++++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 139 ++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 127 ++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 38 ++++
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 21 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 1 +
src/include/utils/rel.h | 1 +
.../regress/expected/collate.icu.utf8.out | 118 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 76 +++++++
18 files changed, 783 insertions(+), 59 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 046b3d37ce..325d894d35 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3558,7 +3558,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8880586c37..cd5c93d4ef 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -724,6 +725,7 @@ index_create(Relation heapRelation,
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
+ List *collations = NIL;
/* constraint flags can only be set when a constraint is requested */
Assert((constr_flags == 0) ||
@@ -1115,21 +1117,44 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /*
+ * Get required distinct dependencies on collations for all index keys.
+ * Collations of directly referenced column in hash indexes can be
+ * skipped is they're deterministic.
+ */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ if ((indexInfo->ii_Am != HASH_AM_OID) ||
+ !get_collation_isdeterministic(colloid))
+ collations = list_append_unique_oid(collations, colloid);
+ }
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ collations = list_concat_unique_oid(collations,
+ GetTypeCollations(att->atttypid,
+ (indexInfo->ii_Am == HASH_AM_OID)));
}
}
+ if (collations)
+ {
+ recordDependencyOnCollations(&myself, collations);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1168,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1262,89 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ if (!version)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3605,6 +3721,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3d2b1cc911..0eaefbd032 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..709e7f5530 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,22 +49,29 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, true);
+ }
}
/*
@@ -69,9 +81,9 @@ recordDependencyOnVersion(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +91,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +110,46 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation needs to be tracked */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;;
+ version = get_collation_version_for_oid(referenced->objectId);
+ }
+ }
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +606,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 8d7572da51..6e91518694 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 597c1241f9..123bfa6c01 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -1501,6 +1503,42 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version;
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index df025a5a30..a2c3d73529 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5620,6 +5621,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..6de17e25e8 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,28 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e1a5ab3df3..2bf6f868a5 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..aa06aca90a 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..6a218d7954 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX idx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX idx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX idx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX idx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX idx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX idx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX idx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX idx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX idx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX idx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX idx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX idx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX idx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX idx14_myrange ON collate_test(myrange);
+CREATE INDEX idx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX idx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX idx16_part ON collate_part_1 (val);
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'idx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+------------------------+------------+--------------
+ idx01_t_en_fr__d_es | "en-x-icu" | some version
+ idx01_t_en_fr__d_es | "es-x-icu" | some version
+ idx01_t_en_fr__d_es | "fr-x-icu" | some version
+ idx02_d_en_fr | "en-x-icu" | some version
+ idx02_d_en_fr | "fr-x-icu" | some version
+ idx03_t_en_fr_ga | "en-x-icu" | some version
+ idx03_t_en_fr_ga | "fr-x-icu" | some version
+ idx03_t_en_fr_ga | "ga-x-icu" | some version
+ idx04_d_en_fr_ga | "en-x-icu" | some version
+ idx04_d_en_fr_ga | "fr-x-icu" | some version
+ idx04_d_en_fr_ga | "ga-x-icu" | some version
+ idx05_d_en_fr_ga_arr | "en-x-icu" | some version
+ idx05_d_en_fr_ga_arr | "fr-x-icu" | some version
+ idx05_d_en_fr_ga_arr | "ga-x-icu" | some version
+ idx06_d_en_fr_ga | "default" | no version
+ idx06_d_en_fr_ga | "en-x-icu" | some version
+ idx06_d_en_fr_ga | "fr-x-icu" | some version
+ idx06_d_en_fr_ga | "ga-x-icu" | some version
+ idx07_d_en_fr_ga | "default" | no version
+ idx07_d_en_fr_ga | "en-x-icu" | some version
+ idx07_d_en_fr_ga | "fr-x-icu" | some version
+ idx07_d_en_fr_ga | "ga-x-icu" | some version
+ idx08_d_en_fr_ga | "en-x-icu" | some version
+ idx08_d_en_fr_ga | "fr-x-icu" | some version
+ idx08_d_en_fr_ga | "ga-x-icu" | some version
+ idx09_d_en_fr_ga | "en-x-icu" | some version
+ idx09_d_en_fr_ga | "fr-x-icu" | some version
+ idx09_d_en_fr_ga | "ga-x-icu" | some version
+ idx10_d_en_fr_ga_es | "en-x-icu" | some version
+ idx10_d_en_fr_ga_es | "es-x-icu" | some version
+ idx10_d_en_fr_ga_es | "fr-x-icu" | some version
+ idx10_d_en_fr_ga_es | "ga-x-icu" | some version
+ idx11_d_es | "default" | no version
+ idx11_d_es | "es-x-icu" | some version
+ idx12_custom | "default" | no version
+ idx12_custom | custom | some version
+ idx13_custom | "default" | no version
+ idx13_custom | custom | some version
+ idx14_myrange | "default" | no version
+ idx15_myrange_en_fr_ga | "en-x-icu" | some version
+ idx15_myrange_en_fr_ga | "fr-x-icu" | some version
+ idx15_myrange_en_fr_ga | "ga-x-icu" | some version
+ idx16_mood | "fr-x-icu" | some version
+ idx16_part | "en-x-icu" | some version
+(44 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'idx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ddf3a63c3..92dcda5bdc 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e870594b5b 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,82 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX idx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX idx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX idx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX idx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX idx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX idx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX idx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX idx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX idx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX idx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX idx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX idx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX idx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX idx14_myrange ON collate_test(myrange);
+CREATE INDEX idx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX idx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX idx16_part ON collate_part_1 (val);
+
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'idx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'idx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
0006-Add-a-new-ALTER-INDEX-name-DEPENDS-ON-COLLATION-name-v7.patchapplication/octet-stream; name=0006-Add-a-new-ALTER-INDEX-name-DEPENDS-ON-COLLATION-name-v7.patchDownload
From 77a5576306b7c35800dd3fd1c42ddf98ee1e60ee Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH 6/6] Add a new ALTER INDEX name DEPENDS ON COLLATION name
CURRENT VERSION
This command allows privileged users to specifify that the currently installed
collation library version, for a specific collation, is binary compatible with
the one that was installed when the specified index was built for the last
time.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 13 ++
src/backend/commands/tablecmds.c | 20 ++-
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 26 +++-
src/include/nodes/parsenodes.h | 5 +-
.../regress/expected/collate.icu.utf8.out | 147 ++++++++++--------
src/test/regress/sql/collate.icu.utf8.sql | 47 +++---
7 files changed, 175 insertions(+), 92 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..f4a4333ab1 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON COLLATION <replaceable class="parameter">collation_name</replaceable> CURRENT VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,18 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>DEPENDS ON COLLATION</literal></term>
+ <listitem>
+ <para>
+ This form update the index existing dependency on a specific collation,
+ to specificy the the currently installed collation version is compatible
+ with the version used the last time the index was built. Be aware that
+ an incorrect use of this form can hide a corruption on the index.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b3cf5641fd..31ab4f5ad4 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -4054,7 +4055,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
break;
case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
* [UNKNOWN VERSION | VERSION ...] */
- if (!IsBinaryUpgrade)
+ if (!IsBinaryUpgrade && cmd->version)
ereport(ERROR,
(errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
(errmsg("command can only be called when server is in binary upgrade mode"))));
@@ -17319,8 +17320,9 @@ index_force_collation_version(const ObjectAddress *otherObject,
/* Execute an ALTER INDEX ... ALTER COLLATION DEPENDS ON ...
*
- * A version has to be provided. If the caller wants to notify that the
- * collation version to depend on is unknown, an empty string is passed.
+ * If the caller wants to notify that the collation version to depend on is
+ * unknown, an empty string is passed. If the collation's current version has
+ * to be used, then NULL is passed.
*/
static void
ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
@@ -17328,14 +17330,20 @@ ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
ObjectAddress object;
NewCollationVersionDependency forced_dependency;
- Assert(version);
-
if (coll == NIL)
forced_dependency.oid = InvalidOid;
else
forced_dependency.oid = get_collation_oid(coll, false);
- forced_dependency.version = version;
+ if (version)
+ forced_dependency.version = version;
+ else
+ {
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+ }
object.classId = RelationRelationId;
object.objectId = rel->rd_id;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 1437123540..cb8c4f6b05 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2596,6 +2596,15 @@ alter_table_cmd:
n->version = "";
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> DEPENDS ON COLLATION ... CURRENT VERSION */
+ | DEPENDS ON COLLATION any_name CURRENT_P VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_DependsOnCollationVersion;
+ n->object = $4;
+ n->version = NULL;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index dc03fbde13..e1b56b8fed 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -807,6 +808,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1697,7 +1712,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "DEPENDS ON COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1743,6 +1758,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> DEPENDS ON COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS", "ON", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> DEPENDS ON COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS", "ON", "COLLATION", MatchAny))
+ COMPLETE_WITH("CURRENT VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index b36824ddfe..a418d43352 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1864,7 +1864,10 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
- char *version; /* version reference for collation dependency */
+ char *version; /* version reference for collation dependency.
+ * An empty string is used to represent an
+ * unknown version, and a NULL pointer is used
+ * to represent the current version */
RoleSpec *newowner;
Node *def; /* definition of new column, index,
* constraint, or parent table */
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 6a218d7954..b35c200b24 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1923,26 +1923,26 @@ CREATE TABLE collate_test (
myrange_en_fr_ga myrange_en_fr_ga,
mood mood
);
-CREATE INDEX idx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
-CREATE INDEX idx02_d_en_fr ON collate_test (d_en_fr);
-CREATE INDEX idx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
-CREATE INDEX idx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
-CREATE INDEX idx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
-CREATE INDEX idx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
-CREATE INDEX idx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
-CREATE INDEX idx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
-CREATE INDEX idx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
-CREATE INDEX idx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
-CREATE INDEX idx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
-CREATE INDEX idx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
-CREATE INDEX idx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
-CREATE INDEX idx14_myrange ON collate_test(myrange);
-CREATE INDEX idx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
-CREATE INDEX idx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
-CREATE INDEX idx16_part ON collate_part_1 (val);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
-- version of default (and libc) collation is only tracked on glibc systems
SELECT objid::regclass, refobjid::regcollation,
CASE
@@ -1953,59 +1953,59 @@ FROM pg_depend d
LEFT JOIN pg_class c ON c.oid = d.objid
WHERE refclassid = 'pg_collation'::regclass
AND coalesce(relkind, 'i') = 'i'
-AND relname LIKE 'idx%'
+AND relname LIKE 'icuidx%'
ORDER BY 1, 2;
- objid | refobjid | version
-------------------------+------------+--------------
- idx01_t_en_fr__d_es | "en-x-icu" | some version
- idx01_t_en_fr__d_es | "es-x-icu" | some version
- idx01_t_en_fr__d_es | "fr-x-icu" | some version
- idx02_d_en_fr | "en-x-icu" | some version
- idx02_d_en_fr | "fr-x-icu" | some version
- idx03_t_en_fr_ga | "en-x-icu" | some version
- idx03_t_en_fr_ga | "fr-x-icu" | some version
- idx03_t_en_fr_ga | "ga-x-icu" | some version
- idx04_d_en_fr_ga | "en-x-icu" | some version
- idx04_d_en_fr_ga | "fr-x-icu" | some version
- idx04_d_en_fr_ga | "ga-x-icu" | some version
- idx05_d_en_fr_ga_arr | "en-x-icu" | some version
- idx05_d_en_fr_ga_arr | "fr-x-icu" | some version
- idx05_d_en_fr_ga_arr | "ga-x-icu" | some version
- idx06_d_en_fr_ga | "default" | no version
- idx06_d_en_fr_ga | "en-x-icu" | some version
- idx06_d_en_fr_ga | "fr-x-icu" | some version
- idx06_d_en_fr_ga | "ga-x-icu" | some version
- idx07_d_en_fr_ga | "default" | no version
- idx07_d_en_fr_ga | "en-x-icu" | some version
- idx07_d_en_fr_ga | "fr-x-icu" | some version
- idx07_d_en_fr_ga | "ga-x-icu" | some version
- idx08_d_en_fr_ga | "en-x-icu" | some version
- idx08_d_en_fr_ga | "fr-x-icu" | some version
- idx08_d_en_fr_ga | "ga-x-icu" | some version
- idx09_d_en_fr_ga | "en-x-icu" | some version
- idx09_d_en_fr_ga | "fr-x-icu" | some version
- idx09_d_en_fr_ga | "ga-x-icu" | some version
- idx10_d_en_fr_ga_es | "en-x-icu" | some version
- idx10_d_en_fr_ga_es | "es-x-icu" | some version
- idx10_d_en_fr_ga_es | "fr-x-icu" | some version
- idx10_d_en_fr_ga_es | "ga-x-icu" | some version
- idx11_d_es | "default" | no version
- idx11_d_es | "es-x-icu" | some version
- idx12_custom | "default" | no version
- idx12_custom | custom | some version
- idx13_custom | "default" | no version
- idx13_custom | custom | some version
- idx14_myrange | "default" | no version
- idx15_myrange_en_fr_ga | "en-x-icu" | some version
- idx15_myrange_en_fr_ga | "fr-x-icu" | some version
- idx15_myrange_en_fr_ga | "ga-x-icu" | some version
- idx16_mood | "fr-x-icu" | some version
- idx16_part | "en-x-icu" | some version
+ objid | refobjid | version
+---------------------------+------------+--------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "es-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | some version
+ icuidx02_d_en_fr | "en-x-icu" | some version
+ icuidx02_d_en_fr | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "en-x-icu" | some version
+ icuidx03_t_en_fr_ga | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "ga-x-icu" | some version
+ icuidx04_d_en_fr_ga | "en-x-icu" | some version
+ icuidx04_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx04_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version
+ icuidx06_d_en_fr_ga | "default" | no version
+ icuidx06_d_en_fr_ga | "en-x-icu" | some version
+ icuidx06_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx06_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx07_d_en_fr_ga | "default" | no version
+ icuidx07_d_en_fr_ga | "en-x-icu" | some version
+ icuidx07_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx07_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx08_d_en_fr_ga | "en-x-icu" | some version
+ icuidx08_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx08_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx09_d_en_fr_ga | "en-x-icu" | some version
+ icuidx09_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx09_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version
+ icuidx11_d_es | "default" | no version
+ icuidx11_d_es | "es-x-icu" | some version
+ icuidx12_custom | "default" | no version
+ icuidx12_custom | custom | some version
+ icuidx13_custom | "default" | no version
+ icuidx13_custom | custom | some version
+ icuidx14_myrange | "default" | no version
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version
+ icuidx16_mood | "fr-x-icu" | some version
+ icuidx17_part | "en-x-icu" | some version
(44 rows)
UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
-AND objid::regclass::text LIKE 'idx%'
+AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
REINDEX TABLE collate_test;
REINDEX TABLE collate_part_0;
@@ -2015,6 +2015,23 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name DEPENDS ON COLLATION name CURRENT VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part DEPENDS ON COLLATION "en-x-icu" CURRENT VERSION;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e870594b5b..885470c7ba 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -747,27 +747,27 @@ CREATE TABLE collate_test (
mood mood
);
-CREATE INDEX idx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
-CREATE INDEX idx02_d_en_fr ON collate_test (d_en_fr);
-CREATE INDEX idx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
-CREATE INDEX idx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
-CREATE INDEX idx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
-CREATE INDEX idx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
-CREATE INDEX idx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
-CREATE INDEX idx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
-CREATE INDEX idx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
-CREATE INDEX idx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
-CREATE INDEX idx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
-CREATE INDEX idx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
-CREATE INDEX idx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
-CREATE INDEX idx14_myrange ON collate_test(myrange);
-CREATE INDEX idx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
-CREATE INDEX idx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
-CREATE INDEX idx16_part ON collate_part_1 (val);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
-- version of default (and libc) collation is only tracked on glibc systems
SELECT objid::regclass, refobjid::regcollation,
@@ -779,12 +779,12 @@ FROM pg_depend d
LEFT JOIN pg_class c ON c.oid = d.objid
WHERE refclassid = 'pg_collation'::regclass
AND coalesce(relkind, 'i') = 'i'
-AND relname LIKE 'idx%'
+AND relname LIKE 'icuidx%'
ORDER BY 1, 2;
UPDATE pg_depend SET refobjversion = 'not a version'
WHERE refclassid = 'pg_collation'::regclass
-AND objid::regclass::text LIKE 'idx%'
+AND objid::regclass::text LIKE 'icuidx%'
AND refobjversion IS NOT NULL;
REINDEX TABLE collate_test;
@@ -793,6 +793,15 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name DEPENDS ON COLLATION name CURRENT VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part DEPENDS ON COLLATION "en-x-icu" CURRENT VERSION;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
Re: Collation versioning
On Wed, Feb 05, 2020 at 05:17:25PM +0100, Julien Rouhaud wrote:
Note that I didn't change any syntax (or switched to native functions
for the binary pg_dump) as it's still not clear to me what exactly
should be implemented.
Hearing no complaints on the suggestions, I'm attaching v8 to address that:
- pg_dump is now using a binary_upgrade_set_index_coll_version() function
rather than plain DDL
- the additional DDL is now of the form:
ALTER INDEX name ALTER COLLATION name REFRESH VERSION
I also added an alternate file for the collate.icu.utf8, so the build farm bot
should turn green for the linux part.
Attachments:
0004-Track-collation-versions-for-indexes-v8.patchtext/plain; charset=us-asciiDownload
From cf3b46a0727366e00683421f45fbd6a228106b53 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 4/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 189 ++++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 139 ++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 127 ++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 38 ++++
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 21 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 1 +
src/include/utils/rel.h | 1 +
.../regress/expected/collate.icu.utf8.out | 118 +++++++++++
...te.icu.utf8.out => collate.icu.utf8_2.out} | 118 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 76 +++++++
19 files changed, 901 insertions(+), 59 deletions(-)
copy src/test/regress/expected/{collate.icu.utf8.out => collate.icu.utf8_2.out} (89%)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e31478bf91..46e9d74a97 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8880586c37..cd5c93d4ef 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -724,6 +725,7 @@ index_create(Relation heapRelation,
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
+ List *collations = NIL;
/* constraint flags can only be set when a constraint is requested */
Assert((constr_flags == 0) ||
@@ -1115,21 +1117,44 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /*
+ * Get required distinct dependencies on collations for all index keys.
+ * Collations of directly referenced column in hash indexes can be
+ * skipped is they're deterministic.
+ */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ if ((indexInfo->ii_Am != HASH_AM_OID) ||
+ !get_collation_isdeterministic(colloid))
+ collations = list_append_unique_oid(collations, colloid);
+ }
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ collations = list_concat_unique_oid(collations,
+ GetTypeCollations(att->atttypid,
+ (indexInfo->ii_Am == HASH_AM_OID)));
}
}
+ if (collations)
+ {
+ recordDependencyOnCollations(&myself, collations);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1168,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1262,89 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ if (!version)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3605,6 +3721,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3d2b1cc911..0eaefbd032 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..709e7f5530 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,22 +49,29 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, true);
+ }
}
/*
@@ -69,9 +81,9 @@ recordDependencyOnVersion(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +91,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +110,46 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation needs to be tracked */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;;
+ version = get_collation_version_for_oid(referenced->objectId);
+ }
+ }
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +606,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 8d7572da51..6e91518694 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 597c1241f9..123bfa6c01 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -1501,6 +1503,42 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version;
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..6de17e25e8 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,28 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e1a5ab3df3..2bf6f868a5 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..aa06aca90a 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..f049189897 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+--------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "es-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | some version
+ icuidx02_d_en_fr | "en-x-icu" | some version
+ icuidx02_d_en_fr | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "en-x-icu" | some version
+ icuidx03_t_en_fr_ga | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "ga-x-icu" | some version
+ icuidx04_d_en_fr_ga | "en-x-icu" | some version
+ icuidx04_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx04_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version
+ icuidx06_d_en_fr_ga | "default" | no version
+ icuidx06_d_en_fr_ga | "en-x-icu" | some version
+ icuidx06_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx06_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx07_d_en_fr_ga | "default" | no version
+ icuidx07_d_en_fr_ga | "en-x-icu" | some version
+ icuidx07_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx07_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx08_d_en_fr_ga | "en-x-icu" | some version
+ icuidx08_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx08_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx09_d_en_fr_ga | "en-x-icu" | some version
+ icuidx09_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx09_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version
+ icuidx11_d_es | "default" | no version
+ icuidx11_d_es | "es-x-icu" | some version
+ icuidx12_custom | "default" | no version
+ icuidx12_custom | custom | some version
+ icuidx13_custom | "default" | no version
+ icuidx13_custom | custom | some version
+ icuidx14_myrange | "default" | no version
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version
+ icuidx16_mood | "fr-x-icu" | some version
+ icuidx17_part | "en-x-icu" | some version
+(44 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8_2.out
similarity index 89%
copy from src/test/regress/expected/collate.icu.utf8.out
copy to src/test/regress/expected/collate.icu.utf8_2.out
index 60d9263a2f..07e76c4f48 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8_2.out
@@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+--------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "es-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | some version
+ icuidx02_d_en_fr | "en-x-icu" | some version
+ icuidx02_d_en_fr | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "en-x-icu" | some version
+ icuidx03_t_en_fr_ga | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "ga-x-icu" | some version
+ icuidx04_d_en_fr_ga | "en-x-icu" | some version
+ icuidx04_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx04_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version
+ icuidx06_d_en_fr_ga | "default" | some version
+ icuidx06_d_en_fr_ga | "en-x-icu" | some version
+ icuidx06_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx06_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx07_d_en_fr_ga | "default" | some version
+ icuidx07_d_en_fr_ga | "en-x-icu" | some version
+ icuidx07_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx07_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx08_d_en_fr_ga | "en-x-icu" | some version
+ icuidx08_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx08_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx09_d_en_fr_ga | "en-x-icu" | some version
+ icuidx09_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx09_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version
+ icuidx11_d_es | "default" | some version
+ icuidx11_d_es | "es-x-icu" | some version
+ icuidx12_custom | "default" | some version
+ icuidx12_custom | custom | some version
+ icuidx13_custom | "default" | some version
+ icuidx13_custom | custom | some version
+ icuidx14_myrange | "default" | some version
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version
+ icuidx16_mood | "fr-x-icu" | some version
+ icuidx17_part | "en-x-icu" | some version
+(44 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ddf3a63c3..92dcda5bdc 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..920a73512f 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,82 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
0001-Remove-pg_collation.collversion-v8.patchtext/plain; charset=us-asciiDownload
From dd68cfb8b213f83536e482ed6c7b6170cb155467 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/6] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
18 files changed, 11 insertions(+), 290 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index a10b66569b..f187a1af65 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index ceda48e0fc..44d82ca0b3 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21103,10 +21103,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 85f726ae06..493aa21a14 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 54ad62bb7f..bf4f793ba2 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3183,16 +3183,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5172,9 +5162,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 5b1ba143b1..4ec777a78c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3269,9 +3261,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 1b0edf5d3d..b6d6a0e239 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10321,21 +10320,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index bb85b5e52a..e57e05b9ee 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1798,10 +1798,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2948,10 +2944,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 25fb7e2ebf..597c1241f9 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ec3e2c63b0..9aa6496814 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13485,7 +13485,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13557,7 +13561,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index df7d1d498c..9a9e145b4c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index da0706add5..c96d027362 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
0002-Add-pg_depend.refobjversion-v8.patchtext/plain; charset=us-asciiDownload
From 0920cb9c88db698efd4e0b4c4b73cc800ccc9f17 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 28 ++++++++++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 3 ++
src/test/regress/expected/misc_sanity.out | 7 ++--
6 files changed, 66 insertions(+), 33 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index c4a4df25b8..78c31baa34 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2681,7 +2684,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f9af245eec..7fdbdf0ae8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 7f1534aebb..e72f85d8f0 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0cd6fcf027..77cf0612ed 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..41efb4a2c8 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
@@ -103,9 +103,10 @@ ORDER BY 1, 2;
pg_class | relacl | aclitem[]
pg_class | reloptions | text[]
pg_class | relpartbound | pg_node_tree
+ pg_depend | refobjversion | text
pg_index | indexprs | pg_node_tree
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
--
2.20.1
0003-Implement-type-regcollation-v8.patchtext/plain; charset=us-asciiDownload
From 988b4a7c43fbeaf644859b31582cf47fd4d992e3 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH 3/6] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
4 files changed, 185 insertions(+)
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 226c904c04..f6503428cc 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6663,6 +6663,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7449,6 +7458,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index fe2c4eabb4..012c17bb68 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
0005-Preserve-index-dependencies-on-collation-during-pg_u-v8.patchtext/plain; charset=us-asciiDownload
From b33dc0533f2285b6e86766b2dc7c1b836b026337 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/6] Preserve index dependencies on collation during
pg_upgrade
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for all or a single collation, for a specific
index.
Also teach pg_dump to call this function for all indexes, including indexes
created for constraints, when run with --binary-upgrade flag.
When pg_upgrade is run against an older version, collation versions are not
known and pg_dump will by default emit an alter index command to mark all
collation versions as unkown. However, it's possible that pg_upgrade is run
without upgrading the underlying collation libraries, so a new option
--collation-binary-compatible is added to avoid this behavior. This will
result in running pg_dump with a new --unknown-collations-binary-compatible
option, that can only be used in binary upgrade mode, to prevent pg_dump from
emitting call to this new function if the dependent collation version is
unknown. Note that if the collation version is known, this flag won't change
the behavior and the previous collation version will be preserved.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/commands/tablecmds.c | 52 +++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
16 files changed, 564 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..b62a1d4eef 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ When upgrading from a PostgreSQL major version 12 or older, or from a
+ PostgreSQL major version that didn't have support for collation
+ versioning to a major version that now supports it, all indexes will be
+ marked as depending on an unknown collation version, as such versions
+ weren't tracked. As a result, numerous warning messages will be
+ emitted as it can be a sign of a corrupted index. If you're not
+ upgrading the collation libraries, and if you're absolutly certain that
+ all existing indexes are compatible with the current collation
+ libraries, you can use this flag to change this behavior and mark all
+ indexes as depending on current collation libraries version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cd5c93d4ef..b595c90ad0 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3097,6 +3097,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b7c8d663fc..facda420f2 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -553,6 +553,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+//static void ATExecDependsOnCollationVersion(Relation rel, List *coll,
+// char *version);
/* ----------------------------------------------------------------
@@ -3871,6 +3873,11 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ /* Only used in binary upgrade mode */
+ //case AT_DependsOnCollationVersion:
+ // cmd_lockmode = AccessExclusiveLock;
+ // break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4038,6 +4045,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ //case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
+ // * [UNKNOWN VERSION | VERSION ...] */
+ // if (!IsBinaryUpgrade)
+ // ereport(ERROR,
+ // (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ // (errmsg("command can only be called when server is in binary upgrade mode"))));
+ // ATSimplePermissions(rel, ATT_INDEX);
+ // /* This command never recurses */
+ // pass = AT_PASS_MISC;
+ // break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4604,6 +4621,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ //case AT_DependsOnCollationVersion:
+ // /* ATPrepCmd ensured it must be an index */
+ // Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ // ATExecDependsOnCollationVersion(rel, cmd->object, cmd->version);
+ // break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17269,3 +17291,33 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION DEPENDS ON ...
+ *
+ * A version has to be provided. If the caller wants to notify that the
+ * collation version to depend on is unknown, an empty string is passed.
+ */
+//static void
+//ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
+//{
+// ObjectAddress object;
+// NewCollationVersionDependency forced_dependency;
+//
+// Assert(version);
+//
+// if (coll == NIL)
+// forced_dependency.oid = InvalidOid;
+// else
+// forced_dependency.oid = get_collation_oid(coll, false);
+//
+// forced_dependency.version = version;
+//
+// object.classId = RelationRelationId;
+// object.objectId = rel->rd_id;
+// object.objectSubId = 0;
+// visitDependentObjects(&object, &index_force_collation_version,
+// &forced_dependency);
+//
+// /* Invalidate the index relcache */
+// CacheInvalidateRelcache(rel);
+//}
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 0d9e55cdcf..ea7c25ee35 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9aa6496814..a09a0cf53d 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -288,6 +289,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -388,6 +391,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -708,6 +712,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6841,7 +6849,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6877,7 +6887,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6902,7 +6967,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6941,7 +7008,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6976,7 +7045,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7007,7 +7078,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7041,7 +7114,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7081,6 +7156,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7106,6 +7183,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16374,10 +16453,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16437,6 +16517,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16466,6 +16550,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18423,6 +18522,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 21004e5078..6d1df24080 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 4a9764c2d2..216a704bb7 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -914,9 +933,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1178,6 +1198,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1203,6 +1224,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1238,6 +1260,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1260,6 +1283,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1281,6 +1305,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1302,6 +1327,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1665,6 +1691,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1679,7 +1706,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2347,6 +2374,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2540,6 +2568,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2607,6 +2636,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2678,6 +2708,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3145,6 +3176,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3160,6 +3192,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3292,16 +3325,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3325,6 +3395,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3375,16 +3449,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3432,6 +3519,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3485,79 +3578,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index eed70fff4f..d36d6a283f 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index b156b516cc..af34dff920 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -294,6 +294,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 6de17e25e8..dc99b49b8e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f6503428cc..bca564644c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10195,6 +10195,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
0006-Add-a-new-ALTER-INDEX-name-ALTER-COLLATION-name-REFR-v8.patchtext/plain; charset=us-asciiDownload
From b9d80e5eff355e930a173c6cf8b03e3a0d623c57 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH 6/6] Add a new ALTER INDEX name ALTER COLLATION name REFRESH
VERSION
This command allows privileged users to specifify that the currently installed
collation library version, for a specific collation, is binary compatible with
the one that was installed when the specified index was built for the last
time.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 13 +++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 90 +++++++++----------
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++
src/bin/psql/tab-complete.c | 26 +++++-
src/include/catalog/index.h | 3 +
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 17 ++++
.../regress/expected/collate.icu.utf8_2.out | 17 ++++
src/test/regress/sql/collate.icu.utf8.sql | 9 ++
11 files changed, 139 insertions(+), 51 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..8661b031e9 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,18 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION</literal></term>
+ <listitem>
+ <para>
+ This form update the index existing dependency on a specific collation,
+ to specificy the the currently installed collation version is compatible
+ with the version used the last time the index was built. Be aware that
+ an incorrect use of this form can hide a corruption on the index.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b595c90ad0..a92977fef7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3097,7 +3097,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index facda420f2..feaea2024e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -553,8 +554,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-//static void ATExecDependsOnCollationVersion(Relation rel, List *coll,
-// char *version);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3873,10 +3873,9 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
- /* Only used in binary upgrade mode */
- //case AT_DependsOnCollationVersion:
- // cmd_lockmode = AccessExclusiveLock;
- // break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
@@ -4045,16 +4044,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
- //case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
- // * [UNKNOWN VERSION | VERSION ...] */
- // if (!IsBinaryUpgrade)
- // ereport(ERROR,
- // (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
- // (errmsg("command can only be called when server is in binary upgrade mode"))));
- // ATSimplePermissions(rel, ATT_INDEX);
- // /* This command never recurses */
- // pass = AT_PASS_MISC;
- // break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ...
+ * REFRESH VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4621,11 +4616,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
- //case AT_DependsOnCollationVersion:
- // /* ATPrepCmd ensured it must be an index */
- // Assert(rel->rd_rel->relkind == RELKIND_INDEX);
- // ATExecDependsOnCollationVersion(rel, cmd->object, cmd->version);
- // break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17292,32 +17287,31 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
}
}
-/* Execute an ALTER INDEX ... ALTER COLLATION DEPENDS ON ...
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
*
- * A version has to be provided. If the caller wants to notify that the
- * collation version to depend on is unknown, an empty string is passed.
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
*/
-//static void
-//ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
-//{
-// ObjectAddress object;
-// NewCollationVersionDependency forced_dependency;
-//
-// Assert(version);
-//
-// if (coll == NIL)
-// forced_dependency.oid = InvalidOid;
-// else
-// forced_dependency.oid = get_collation_oid(coll, false);
-//
-// forced_dependency.version = version;
-//
-// object.classId = RelationRelationId;
-// object.objectId = rel->rd_id;
-// object.objectSubId = 0;
-// visitDependentObjects(&object, &index_force_collation_version,
-// &forced_dependency);
-//
-// /* Invalidate the index relcache */
-// CacheInvalidateRelcache(rel);
-//}
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index bf4f793ba2..57c87ded58 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3174,6 +3174,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index b6d6a0e239..886d065a8e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2569,6 +2569,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index dc03fbde13..4cc1646763 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -807,6 +808,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1697,7 +1712,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1743,6 +1758,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 69d163f3cc..065488374e 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index c96d027362..9f637597a7 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index f049189897..2902160688 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2015,6 +2015,23 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/collate.icu.utf8_2.out b/src/test/regress/expected/collate.icu.utf8_2.out
index 07e76c4f48..33a56af104 100644
--- a/src/test/regress/expected/collate.icu.utf8_2.out
+++ b/src/test/regress/expected/collate.icu.utf8_2.out
@@ -2015,6 +2015,23 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 920a73512f..10329e3498 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -793,6 +793,15 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
Re: Collation versioning
On Wed, 2020-02-12 at 20:13 +0100, Julien Rouhaud wrote:
On Wed, Feb 05, 2020 at 05:17:25PM +0100, Julien Rouhaud wrote:
Note that I didn't change any syntax (or switched to native functions
for the binary pg_dump) as it's still not clear to me what exactly
should be implemented.Hearing no complaints on the suggestions, I'm attaching v8 to address that:
- pg_dump is now using a binary_upgrade_set_index_coll_version() function
rather than plain DDL
- the additional DDL is now of the form:
ALTER INDEX name ALTER COLLATION name REFRESH VERSIONI also added an alternate file for the collate.icu.utf8, so the build farm bot
should turn green for the linux part.
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..8661b031e9 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -109,6 +110,18 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>+ <varlistentry>
+ <term><literal>ALTER COLLATION</literal></term>
+ <listitem>
+ <para>
+ This form update the index existing dependency on a specific collation,
+ to specificy the the currently installed collation version is compatible
+ with the version used the last time the index was built. Be aware that
+ an incorrect use of this form can hide a corruption on the index.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>This description could do with some love. Perhaps:
This command declares that the index is compatible with the currently
installed version of the collations that determine its order. It is used
to silence warnings caused by collation
version incompatibilities and
should be called after rebuilding the index or otherwise verifying its
consistency. Be aware that incorrect use of this command can hide
index corruption.
I didn't study the patch in detail, but do I get it right that there will be no
warnings about version incompatibilities with libc collations?
Yours,
Laurenz Albe
Re: Collation versioning
On Wed, Feb 12, 2020 at 08:55:06PM +0100, Laurenz Albe wrote:
On Wed, 2020-02-12 at 20:13 +0100, Julien Rouhaud wrote:
On Wed, Feb 05, 2020 at 05:17:25PM +0100, Julien Rouhaud wrote:
Note that I didn't change any syntax (or switched to native functions
for the binary pg_dump) as it's still not clear to me what exactly
should be implemented.Hearing no complaints on the suggestions, I'm attaching v8 to address that:
- pg_dump is now using a binary_upgrade_set_index_coll_version() function
rather than plain DDL
- the additional DDL is now of the form:
ALTER INDEX name ALTER COLLATION name REFRESH VERSIONI also added an alternate file for the collate.icu.utf8, so the build farm bot
should turn green for the linux part.diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml index 6d34dbb74e..8661b031e9 100644 --- a/doc/src/sgml/ref/alter_index.sgml +++ b/doc/src/sgml/ref/alter_index.sgml @@ -109,6 +110,18 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable> </listitem> </varlistentry>+ <varlistentry> + <term><literal>ALTER COLLATION</literal></term> + <listitem> + <para> + This form update the index existing dependency on a specific collation, + to specificy the the currently installed collation version is compatible + with the version used the last time the index was built. Be aware that + an incorrect use of this form can hide a corruption on the index. + </para> + </listitem> + </varlistentry> + <varlistentry> <term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term> <listitem>This description could do with some love. Perhaps:
This command declares that the index is compatible with the currently
installed version of the collations that determine its order. It is used
to silence warnings caused by collation
version incompatibilities and
should be called after rebuilding the index or otherwise verifying its
consistency. Be aware that incorrect use of this command can hide
index corruption.
Thanks a lot, that's indeed way better! I'll add it in the round of patch.
I didn't study the patch in detail, but do I get it right that there will be no
warnings about version incompatibilities with libc collations?
No, libc is also be supported (including the default collation), as long as we
have a way to get the version. Unfortunately, that means only linux/glibc. I
think that there was some previous discussion to work around that limitation
for other systems, using some kind of hash of the underlying collation files,
as Peter mentioned recently, but that's not part of this patchset.
Re: Collation versioning
On Thu, Feb 13, 2020 at 9:16 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Wed, Feb 12, 2020 at 08:55:06PM +0100, Laurenz Albe wrote:
I didn't study the patch in detail, but do I get it right that there will be no
warnings about version incompatibilities with libc collations?No, libc is also be supported (including the default collation), as long as we
have a way to get the version. Unfortunately, that means only linux/glibc. I
think that there was some previous discussion to work around that limitation
for other systems, using some kind of hash of the underlying collation files,
as Peter mentioned recently, but that's not part of this patchset.
Yeah, this is about the cataloguing infrastructure part, to get the
model and mechanisms right. To actually get version information from
the underlying collation provider, there will need to be a series of
per-OS projects. For glibc right now, it's done, but we just use the
whole glibc version as a proxy (sadly we know this can give false
positives and false negatives, but is expected to work a lot better
than nothing). I hope we can get a proper CLDR version out of that
library one day. For FreeBSD libc, I have patches, I just need more
round tuits. For Windows, there is
https://commitfest.postgresql.org/27/2351/ which I'm planning to
commit soonish, after some more thought about the double-version
thing. Then there is the "run a user-supplied script that gives me a
version" concept, which might work and perhaps allow package
maintainers to supply a script that works on each system. Again,
that'd be a separate project. I guess there will probably always be
some OSes that we can't get the data from so we'll probably always
have to support "don't know" mode.
Re: Collation versioning
On Thu, Feb 13, 2020 at 09:44:40AM +1300, Thomas Munro wrote:
On Thu, Feb 13, 2020 at 9:16 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Wed, Feb 12, 2020 at 08:55:06PM +0100, Laurenz Albe wrote:
I didn't study the patch in detail, but do I get it right that there will be no
warnings about version incompatibilities with libc collations?No, libc is also be supported (including the default collation), as long as we
have a way to get the version. Unfortunately, that means only linux/glibc. I
think that there was some previous discussion to work around that limitation
for other systems, using some kind of hash of the underlying collation files,
as Peter mentioned recently, but that's not part of this patchset.Yeah, this is about the cataloguing infrastructure part, to get the
model and mechanisms right. To actually get version information from
the underlying collation provider, there will need to be a series of
per-OS projects. For glibc right now, it's done, but we just use the
whole glibc version as a proxy (sadly we know this can give false
positives and false negatives, but is expected to work a lot better
than nothing). I hope we can get a proper CLDR version out of that
library one day. For FreeBSD libc, I have patches, I just need more
round tuits. For Windows, there is
https://commitfest.postgresql.org/27/2351/ which I'm planning to
commit soonish, after some more thought about the double-version
thing. Then there is the "run a user-supplied script that gives me a
version" concept, which might work and perhaps allow package
maintainers to supply a script that works on each system. Again,
that'd be a separate project.
Thanks for working on that, it'll be great improvements!
I guess there will probably always be
some OSes that we can't get the data from so we'll probably always
have to support "don't know" mode.
I realized this morning that I didn't add test to validate that we emit the
expected warnings in case of version mismatch. While adding some, I found that
for the unknown versino, my code was actually testing the "versioning support
for that lib on that system is now supported" and not "you apparently upgraded
from pre-v13 with supported collation library versioning, and the version was
marked as unknown". Attached v9 fixes the test to handle both cases.
I also added TAP regression tests for version mismatch checking in the
src/test/locale tests. This subdirectory wasn't included by default, probably
because there was no "check" or "installcheck" target so building test-ctype
was pointless, it's now included by default.
Attachments:
0001-Remove-pg_collation.collversion-v9.patchtext/plain; charset=us-asciiDownload
From 68bb757dd159c8c48f7f97c1f46d40213b3ba94a Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/6] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
18 files changed, 11 insertions(+), 290 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index a10b66569b..f187a1af65 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index ceda48e0fc..44d82ca0b3 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21103,10 +21103,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 85f726ae06..493aa21a14 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 54ad62bb7f..bf4f793ba2 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3183,16 +3183,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5172,9 +5162,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 5b1ba143b1..4ec777a78c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3269,9 +3261,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 1b0edf5d3d..b6d6a0e239 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10321,21 +10320,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index bb85b5e52a..e57e05b9ee 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1798,10 +1798,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2948,10 +2944,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 25fb7e2ebf..597c1241f9 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ec3e2c63b0..9aa6496814 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13485,7 +13485,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13557,7 +13561,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index df7d1d498c..9a9e145b4c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index da0706add5..c96d027362 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
0002-Add-pg_depend.refobjversion-v9.patchtext/plain; charset=us-asciiDownload
From e031cebeacd02ebd759dea7d8a1503c0eb1be28d Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 28 ++++++++++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 3 ++
src/test/regress/expected/misc_sanity.out | 7 ++--
6 files changed, 66 insertions(+), 33 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index c4a4df25b8..78c31baa34 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2681,7 +2684,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f9af245eec..7fdbdf0ae8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 7f1534aebb..e72f85d8f0 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0cd6fcf027..77cf0612ed 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..41efb4a2c8 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
@@ -103,9 +103,10 @@ ORDER BY 1, 2;
pg_class | relacl | aclitem[]
pg_class | reloptions | text[]
pg_class | relpartbound | pg_node_tree
+ pg_depend | refobjversion | text
pg_index | indexprs | pg_node_tree
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
--
2.20.1
0003-Implement-type-regcollation-v9.patchtext/plain; charset=us-asciiDownload
From 501b7b8e1ff2f46fb0b66a13178f8271087dec91 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH 3/6] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
4 files changed, 185 insertions(+)
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 226c904c04..f6503428cc 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6663,6 +6663,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7449,6 +7458,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index fe2c4eabb4..012c17bb68 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
0004-Track-collation-versions-for-indexes-v9.patchtext/plain; charset=us-asciiDownload
From fcf983953f4ef22ab6b8e900a1a45e70473e0d63 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 4/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 189 ++++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 143 ++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 127 ++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 38 ++++
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 21 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 1 +
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 118 +++++++++++
...te.icu.utf8.out => collate.icu.utf8_2.out} | 118 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 76 +++++++
23 files changed, 992 insertions(+), 61 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
copy src/test/regress/expected/{collate.icu.utf8.out => collate.icu.utf8_2.out} (89%)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e31478bf91..46e9d74a97 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8880586c37..4fb8ce76d0 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -724,6 +725,7 @@ index_create(Relation heapRelation,
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
+ List *collations = NIL;
/* constraint flags can only be set when a constraint is requested */
Assert((constr_flags == 0) ||
@@ -1115,21 +1117,44 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /*
+ * Get required distinct dependencies on collations for all index keys.
+ * Collations of directly referenced column in hash indexes can be
+ * skipped is they're deterministic.
+ */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ if ((indexInfo->ii_Am != HASH_AM_OID) ||
+ !get_collation_isdeterministic(colloid))
+ collations = list_append_unique_oid(collations, colloid);
+ }
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ collations = list_concat_unique_oid(collations,
+ GetTypeCollations(att->atttypid,
+ (indexInfo->ii_Am == HASH_AM_OID)));
}
}
+ if (collations)
+ {
+ recordDependencyOnCollations(&myself, collations);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1168,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1262,93 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || strcmp(version, "") == 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3605,6 +3725,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3d2b1cc911..0eaefbd032 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..709e7f5530 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,22 +49,29 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, true);
+ }
}
/*
@@ -69,9 +81,9 @@ recordDependencyOnVersion(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +91,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +110,46 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation needs to be tracked */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;;
+ version = get_collation_version_for_oid(referenced->objectId);
+ }
+ }
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +606,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 8d7572da51..6e91518694 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 597c1241f9..123bfa6c01 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -1501,6 +1503,42 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version;
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..6de17e25e8 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,28 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e1a5ab3df3..2bf6f868a5 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..aa06aca90a 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..f049189897 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+--------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "es-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | some version
+ icuidx02_d_en_fr | "en-x-icu" | some version
+ icuidx02_d_en_fr | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "en-x-icu" | some version
+ icuidx03_t_en_fr_ga | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "ga-x-icu" | some version
+ icuidx04_d_en_fr_ga | "en-x-icu" | some version
+ icuidx04_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx04_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version
+ icuidx06_d_en_fr_ga | "default" | no version
+ icuidx06_d_en_fr_ga | "en-x-icu" | some version
+ icuidx06_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx06_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx07_d_en_fr_ga | "default" | no version
+ icuidx07_d_en_fr_ga | "en-x-icu" | some version
+ icuidx07_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx07_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx08_d_en_fr_ga | "en-x-icu" | some version
+ icuidx08_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx08_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx09_d_en_fr_ga | "en-x-icu" | some version
+ icuidx09_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx09_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version
+ icuidx11_d_es | "default" | no version
+ icuidx11_d_es | "es-x-icu" | some version
+ icuidx12_custom | "default" | no version
+ icuidx12_custom | custom | some version
+ icuidx13_custom | "default" | no version
+ icuidx13_custom | custom | some version
+ icuidx14_myrange | "default" | no version
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version
+ icuidx16_mood | "fr-x-icu" | some version
+ icuidx17_part | "en-x-icu" | some version
+(44 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8_2.out
similarity index 89%
copy from src/test/regress/expected/collate.icu.utf8.out
copy to src/test/regress/expected/collate.icu.utf8_2.out
index 60d9263a2f..07e76c4f48 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8_2.out
@@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+--------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "es-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | some version
+ icuidx02_d_en_fr | "en-x-icu" | some version
+ icuidx02_d_en_fr | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "en-x-icu" | some version
+ icuidx03_t_en_fr_ga | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "ga-x-icu" | some version
+ icuidx04_d_en_fr_ga | "en-x-icu" | some version
+ icuidx04_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx04_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version
+ icuidx06_d_en_fr_ga | "default" | some version
+ icuidx06_d_en_fr_ga | "en-x-icu" | some version
+ icuidx06_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx06_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx07_d_en_fr_ga | "default" | some version
+ icuidx07_d_en_fr_ga | "en-x-icu" | some version
+ icuidx07_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx07_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx08_d_en_fr_ga | "en-x-icu" | some version
+ icuidx08_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx08_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx09_d_en_fr_ga | "en-x-icu" | some version
+ icuidx09_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx09_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version
+ icuidx11_d_es | "default" | some version
+ icuidx11_d_es | "es-x-icu" | some version
+ icuidx12_custom | "default" | some version
+ icuidx12_custom | custom | some version
+ icuidx13_custom | "default" | some version
+ icuidx13_custom | custom | some version
+ icuidx14_myrange | "default" | some version
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version
+ icuidx16_mood | "fr-x-icu" | some version
+ icuidx17_part | "en-x-icu" | some version
+(44 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ddf3a63c3..92dcda5bdc 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..920a73512f 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,82 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
0005-Preserve-index-dependencies-on-collation-during-pg_u-v9.patchtext/plain; charset=us-asciiDownload
From aa2561b5aab8d22585b8a977c81d854221746902 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/6] Preserve index dependencies on collation during
pg_upgrade
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for all or a single collation, for a specific
index.
Also teach pg_dump to call this function for all indexes, including indexes
created for constraints, when run with --binary-upgrade flag.
When pg_upgrade is run against an older version, collation versions are not
known and pg_dump will by default emit an alter index command to mark all
collation versions as unkown. However, it's possible that pg_upgrade is run
without upgrading the underlying collation libraries, so a new option
--collation-binary-compatible is added to avoid this behavior. This will
result in running pg_dump with a new --unknown-collations-binary-compatible
option, that can only be used in binary upgrade mode, to prevent pg_dump from
emitting call to this new function if the dependent collation version is
unknown. Note that if the collation version is known, this flag won't change
the behavior and the previous collation version will be preserved.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/commands/tablecmds.c | 52 +++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
16 files changed, 564 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..b62a1d4eef 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ When upgrading from a PostgreSQL major version 12 or older, or from a
+ PostgreSQL major version that didn't have support for collation
+ versioning to a major version that now supports it, all indexes will be
+ marked as depending on an unknown collation version, as such versions
+ weren't tracked. As a result, numerous warning messages will be
+ emitted as it can be a sign of a corrupted index. If you're not
+ upgrading the collation libraries, and if you're absolutly certain that
+ all existing indexes are compatible with the current collation
+ libraries, you can use this flag to change this behavior and mark all
+ indexes as depending on current collation libraries version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4fb8ce76d0..763be878bd 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3101,6 +3101,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b7c8d663fc..facda420f2 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -553,6 +553,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+//static void ATExecDependsOnCollationVersion(Relation rel, List *coll,
+// char *version);
/* ----------------------------------------------------------------
@@ -3871,6 +3873,11 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ /* Only used in binary upgrade mode */
+ //case AT_DependsOnCollationVersion:
+ // cmd_lockmode = AccessExclusiveLock;
+ // break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4038,6 +4045,16 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ //case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
+ // * [UNKNOWN VERSION | VERSION ...] */
+ // if (!IsBinaryUpgrade)
+ // ereport(ERROR,
+ // (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
+ // (errmsg("command can only be called when server is in binary upgrade mode"))));
+ // ATSimplePermissions(rel, ATT_INDEX);
+ // /* This command never recurses */
+ // pass = AT_PASS_MISC;
+ // break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4604,6 +4621,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ //case AT_DependsOnCollationVersion:
+ // /* ATPrepCmd ensured it must be an index */
+ // Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ // ATExecDependsOnCollationVersion(rel, cmd->object, cmd->version);
+ // break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17269,3 +17291,33 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION DEPENDS ON ...
+ *
+ * A version has to be provided. If the caller wants to notify that the
+ * collation version to depend on is unknown, an empty string is passed.
+ */
+//static void
+//ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
+//{
+// ObjectAddress object;
+// NewCollationVersionDependency forced_dependency;
+//
+// Assert(version);
+//
+// if (coll == NIL)
+// forced_dependency.oid = InvalidOid;
+// else
+// forced_dependency.oid = get_collation_oid(coll, false);
+//
+// forced_dependency.version = version;
+//
+// object.classId = RelationRelationId;
+// object.objectId = rel->rd_id;
+// object.objectSubId = 0;
+// visitDependentObjects(&object, &index_force_collation_version,
+// &forced_dependency);
+//
+// /* Invalidate the index relcache */
+// CacheInvalidateRelcache(rel);
+//}
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 0d9e55cdcf..ea7c25ee35 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9aa6496814..a09a0cf53d 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -288,6 +289,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -388,6 +391,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -708,6 +712,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6841,7 +6849,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6877,7 +6887,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6902,7 +6967,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6941,7 +7008,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6976,7 +7045,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7007,7 +7078,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7041,7 +7114,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7081,6 +7156,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7106,6 +7183,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16374,10 +16453,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16437,6 +16517,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16466,6 +16550,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18423,6 +18522,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 21004e5078..6d1df24080 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 4a9764c2d2..216a704bb7 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -914,9 +933,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1178,6 +1198,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1203,6 +1224,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1238,6 +1260,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1260,6 +1283,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1281,6 +1305,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1302,6 +1327,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1665,6 +1691,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1679,7 +1706,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2347,6 +2374,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2540,6 +2568,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2607,6 +2636,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2678,6 +2708,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3145,6 +3176,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3160,6 +3192,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3292,16 +3325,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3325,6 +3395,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3375,16 +3449,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3432,6 +3519,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3485,79 +3578,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index eed70fff4f..d36d6a283f 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index b156b516cc..af34dff920 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -294,6 +294,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 6de17e25e8..dc99b49b8e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f6503428cc..bca564644c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10195,6 +10195,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
0006-Add-a-new-ALTER-INDEX-name-ALTER-COLLATION-name-REFR-v9.patchtext/plain; charset=us-asciiDownload
From 6302e4a65cb8956e2ae929d3ff559fecf8c47ee4 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH 6/6] Add a new ALTER INDEX name ALTER COLLATION name REFRESH
VERSION
This command allows privileged users to specifify that the currently installed
collation library version, for a specific collation, is binary compatible with
the one that was installed when the specified index was built for the last
time.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 15 ++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 90 +++++++++----------
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++
src/bin/psql/tab-complete.c | 26 +++++-
src/include/catalog/index.h | 3 +
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 17 ++++
.../regress/expected/collate.icu.utf8_2.out | 17 ++++
src/test/regress/sql/collate.icu.utf8.sql | 9 ++
11 files changed, 141 insertions(+), 51 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..2fb3d69042 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,20 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of the collations that determine its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be called after rebuilding the index or otherwise verifying its
+ consistency. Be aware that incorrect use of this command can hide index
+ corruption.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 763be878bd..df600430c5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3101,7 +3101,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index facda420f2..feaea2024e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -553,8 +554,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-//static void ATExecDependsOnCollationVersion(Relation rel, List *coll,
-// char *version);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3873,10 +3873,9 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
- /* Only used in binary upgrade mode */
- //case AT_DependsOnCollationVersion:
- // cmd_lockmode = AccessExclusiveLock;
- // break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
@@ -4045,16 +4044,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
- //case AT_DependsOnCollationVersion: /* DEPENDS ON COLLATION ...
- // * [UNKNOWN VERSION | VERSION ...] */
- // if (!IsBinaryUpgrade)
- // ereport(ERROR,
- // (errcode(ERRCODE_CANT_CHANGE_RUNTIME_PARAM),
- // (errmsg("command can only be called when server is in binary upgrade mode"))));
- // ATSimplePermissions(rel, ATT_INDEX);
- // /* This command never recurses */
- // pass = AT_PASS_MISC;
- // break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ...
+ * REFRESH VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4621,11 +4616,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
- //case AT_DependsOnCollationVersion:
- // /* ATPrepCmd ensured it must be an index */
- // Assert(rel->rd_rel->relkind == RELKIND_INDEX);
- // ATExecDependsOnCollationVersion(rel, cmd->object, cmd->version);
- // break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17292,32 +17287,31 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
}
}
-/* Execute an ALTER INDEX ... ALTER COLLATION DEPENDS ON ...
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
*
- * A version has to be provided. If the caller wants to notify that the
- * collation version to depend on is unknown, an empty string is passed.
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
*/
-//static void
-//ATExecDependsOnCollationVersion(Relation rel, List *coll, char *version)
-//{
-// ObjectAddress object;
-// NewCollationVersionDependency forced_dependency;
-//
-// Assert(version);
-//
-// if (coll == NIL)
-// forced_dependency.oid = InvalidOid;
-// else
-// forced_dependency.oid = get_collation_oid(coll, false);
-//
-// forced_dependency.version = version;
-//
-// object.classId = RelationRelationId;
-// object.objectId = rel->rd_id;
-// object.objectSubId = 0;
-// visitDependentObjects(&object, &index_force_collation_version,
-// &forced_dependency);
-//
-// /* Invalidate the index relcache */
-// CacheInvalidateRelcache(rel);
-//}
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index bf4f793ba2..57c87ded58 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3174,6 +3174,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index b6d6a0e239..886d065a8e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2569,6 +2569,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index dc03fbde13..4cc1646763 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -807,6 +808,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1697,7 +1712,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1743,6 +1758,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 69d163f3cc..065488374e 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index c96d027362..9f637597a7 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index f049189897..2902160688 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2015,6 +2015,23 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/collate.icu.utf8_2.out b/src/test/regress/expected/collate.icu.utf8_2.out
index 07e76c4f48..33a56af104 100644
--- a/src/test/regress/expected/collate.icu.utf8_2.out
+++ b/src/test/regress/expected/collate.icu.utf8_2.out
@@ -2015,6 +2015,23 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 920a73512f..10329e3498 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -793,6 +793,15 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
Re: Collation versioning
On Thu, Feb 13, 2020 at 8:13 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
Hearing no complaints on the suggestions, I'm attaching v8 to address that:
- pg_dump is now using a binary_upgrade_set_index_coll_version() function
rather than plain DDL
- the additional DDL is now of the form:
ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+1
A couple of thoughts:
@@ -1115,21 +1117,44 @@ index_create(Relation heapRelation,
...
+ /*
+ * Get required distinct dependencies on collations
for all index keys.
+ * Collations of directly referenced column in hash
indexes can be
+ * skipped is they're deterministic.
+ */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ if ((indexInfo->ii_Am != HASH_AM_OID) ||
+
!get_collation_isdeterministic(colloid))I still don't like the way catalog/index.c has hard-coded knowledge of
HASH_AM_OID here. Although it errs on the side of the assuming that
there *is* a version dependency (good), there is already another AM in
the tree that could safely skip it for deterministic collations AFAIK:
Bloom indexes. I suppose that any extension AM that is doing some
kind of hashing would also like to be able to be able to opt out of
collation version checking, when that is safe. The question is how to
model that in our system...
One way would be for each AM to declare whether it is affected by
collations; the answer could be yes/maybe (default), no,
only-non-deterministic-ones. But that still feels like the wrong
level, not taking advantage of knowledge about operators.
A better way might be to make declarations about that sort of thing in
the catalog, somewhere in the vicinity of the operator classes, or
maybe just to have hard coded knowledge about operator classes (ie
making declarations in the manual about what eg hash functions are
allowed to consult and when), and then check which of those an index
depends on. I am not sure what would be best, I'd need to spend some
time studying the am operator system.
Perhaps for the first version of this feature, we should just add a
new local function
index_can_skip_collation_version_dependency(indexInfo, colloid) to
encapsulate your existing logic, but add a comment that in future we
might be able to support skipping in more cases through analysis of
the catalogs.
+ <varlistentry>
+ <term><literal>ALTER COLLATION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of the collations that determine its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be called after rebuilding the index or otherwise verifying its
+ consistency. Be aware that incorrect use of this command can hide index
+ corruption.
+ </para>
+ </listitem>
+ </varlistentry>This sounds like something that you need to do after you reindex, but
that's not true, is it? This is something you can do *instead* of
reindex, to make it shut up about versions. Shouldn't it be something
like "... should be issued only if the ordering is known not to have
changed since the index was built"?
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+Would it be better to put refobjversion = 'not a version' in the
SELECT list, instead of the WHERE clause, with a WHERE clause that
hits that one row, so that we can see that the row still exists after
the REFRESH VERSION (while still hiding the unstable version string)?
Re: Collation versioning
On Mon, Feb 17, 2020 at 5:58 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Thu, Feb 13, 2020 at 8:13 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
Hearing no complaints on the suggestions, I'm attaching v8 to address that:
- pg_dump is now using a binary_upgrade_set_index_coll_version() function
rather than plain DDL
- the additional DDL is now of the form:
ALTER INDEX name ALTER COLLATION name REFRESH VERSION+1
A couple of thoughts:
@@ -1115,21 +1117,44 @@ index_create(Relation heapRelation, ... + /* + * Get required distinct dependencies on collations for all index keys. + * Collations of directly referenced column in hash indexes can be + * skipped is they're deterministic. + */ for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++) { - if (OidIsValid(collationObjectId[i]) && - collationObjectId[i] != DEFAULT_COLLATION_OID) + Oid colloid = collationObjectId[i]; + + if (OidIsValid(colloid)) { - referenced.classId = CollationRelationId; - referenced.objectId = collationObjectId[i]; - referenced.objectSubId = 0; + if ((indexInfo->ii_Am != HASH_AM_OID) || + !get_collation_isdeterministic(colloid))I still don't like the way catalog/index.c has hard-coded knowledge of
HASH_AM_OID here. Although it errs on the side of the assuming that
there *is* a version dependency (good)
Oh, but it also means that it fails to create a versionless
dependency, which is totally wrong. What we should do is instead
setup a "track_version" flag to pass down.
It also means that the current way I handled unknown version (empty
string) vs unknown collation lib version (null) will be problematic,
both for runtime check and pg_upgrade. I think we should record an
empty string for both cases, and keep NULL for when explicitly no
version has to be recorded (whether it's not a dependency on
collation, or because the depender doesn't care about version). It
also mean that I'm missing regression tests using such an access
method.
there is already another AM in
the tree that could safely skip it for deterministic collations AFAIK:
Bloom indexes. I suppose that any extension AM that is doing some
kind of hashing would also like to be able to be able to opt out of
collation version checking, when that is safe. The question is how to
model that in our system...
Oh indeed.
One way would be for each AM to declare whether it is affected by
collations; the answer could be yes/maybe (default), no,
only-non-deterministic-ones. But that still feels like the wrong
level, not taking advantage of knowledge about operators.
On the other hand, would it be possible that some AM only supports
collation-dependency-free operators while still internally relying on
a stable sort order?
A better way might be to make declarations about that sort of thing in
the catalog, somewhere in the vicinity of the operator classes, or
maybe just to have hard coded knowledge about operator classes (ie
making declarations in the manual about what eg hash functions are
allowed to consult and when), and then check which of those an index
depends on. I am not sure what would be best, I'd need to spend some
time studying the am operator system.
I think this will be required at some point anyway, if we want to
eventually avoid warning about possible corruption when an
expression/where clause isn't depending on the collation ordering.
Perhaps for the first version of this feature, we should just add a
new local function
index_can_skip_collation_version_dependency(indexInfo, colloid) to
encapsulate your existing logic, but add a comment that in future we
might be able to support skipping in more cases through analysis of
the catalogs.
That'd be convenient, but would also break extensibility as bloom
would get a preferential treatment (even if such AM doesn't already
exist).
+ <varlistentry> + <term><literal>ALTER COLLATION</literal></term> + <listitem> + <para> + This command declares that the index is compatible with the currently + installed version of the collations that determine its order. It is used + to silence warnings caused by collation version incompatibilities and + should be called after rebuilding the index or otherwise verifying its + consistency. Be aware that incorrect use of this command can hide index + corruption. + </para> + </listitem> + </varlistentry>This sounds like something that you need to do after you reindex, but
that's not true, is it? This is something you can do *instead* of
reindex, to make it shut up about versions. Shouldn't it be something
like "... should be issued only if the ordering is known not to have
changed since the index was built"?
Indeed. We should also probably explicitly mention that if the
situation is unknown, REINDEX is the safe alternative to choose.
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION +UPDATE pg_depend SET refobjversion = 'not a version' +WHERE refclassid = 'pg_collation'::regclass +AND objid::regclass::text = 'icuidx17_part' +AND refobjversion IS NOT NULL; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + objid +--------------- + icuidx17_part +(1 row) + +ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + objid +------- +(0 rows) +Would it be better to put refobjversion = 'not a version' in the
SELECT list, instead of the WHERE clause, with a WHERE clause that
hits that one row, so that we can see that the row still exists after
the REFRESH VERSION (while still hiding the unstable version string)?
Agreed, I'll change that.
Re: Collation versioning
On Tue, Feb 18, 2020 at 2:53 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Mon, Feb 17, 2020 at 5:58 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Thu, Feb 13, 2020 at 8:13 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
Hearing no complaints on the suggestions, I'm attaching v8 to address that:
- pg_dump is now using a binary_upgrade_set_index_coll_version() function
rather than plain DDL
- the additional DDL is now of the form:
ALTER INDEX name ALTER COLLATION name REFRESH VERSION+1
A couple of thoughts:
@@ -1115,21 +1117,44 @@ index_create(Relation heapRelation, ... + /* + * Get required distinct dependencies on collations for all index keys. + * Collations of directly referenced column in hash indexes can be + * skipped is they're deterministic. + */ for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++) { - if (OidIsValid(collationObjectId[i]) && - collationObjectId[i] != DEFAULT_COLLATION_OID) + Oid colloid = collationObjectId[i]; + + if (OidIsValid(colloid)) { - referenced.classId = CollationRelationId; - referenced.objectId = collationObjectId[i]; - referenced.objectSubId = 0; + if ((indexInfo->ii_Am != HASH_AM_OID) || + !get_collation_isdeterministic(colloid))I still don't like the way catalog/index.c has hard-coded knowledge of
HASH_AM_OID here. Although it errs on the side of the assuming that
there *is* a version dependency (good)Oh, but it also means that it fails to create a versionless
dependency, which is totally wrong. What we should do is instead
setup a "track_version" flag to pass down.It also means that the current way I handled unknown version (empty
string) vs unknown collation lib version (null) will be problematic,
both for runtime check and pg_upgrade. I think we should record an
empty string for both cases, and keep NULL for when explicitly no
version has to be recorded (whether it's not a dependency on
collation, or because the depender doesn't care about version).
Fixed this way.
It
also mean that I'm missing regression tests using such an access
method.
Done
there is already another AM in
the tree that could safely skip it for deterministic collations AFAIK:
Bloom indexes. I suppose that any extension AM that is doing some
kind of hashing would also like to be able to be able to opt out of
collation version checking, when that is safe. The question is how to
model that in our system...Oh indeed.
One way would be for each AM to declare whether it is affected by
collations; the answer could be yes/maybe (default), no,
only-non-deterministic-ones. But that still feels like the wrong
level, not taking advantage of knowledge about operators.On the other hand, would it be possible that some AM only supports
collation-dependency-free operators while still internally relying on
a stable sort order?A better way might be to make declarations about that sort of thing in
the catalog, somewhere in the vicinity of the operator classes, or
maybe just to have hard coded knowledge about operator classes (ie
making declarations in the manual about what eg hash functions are
allowed to consult and when), and then check which of those an index
depends on. I am not sure what would be best, I'd need to spend some
time studying the am operator system.I think this will be required at some point anyway, if we want to
eventually avoid warning about possible corruption when an
expression/where clause isn't depending on the collation ordering.Perhaps for the first version of this feature, we should just add a
new local function
index_can_skip_collation_version_dependency(indexInfo, colloid) to
encapsulate your existing logic, but add a comment that in future we
might be able to support skipping in more cases through analysis of
the catalogs.That'd be convenient, but would also break extensibility as bloom
would get a preferential treatment (even if such AM doesn't already
exist).
I added an index_depends_stable_coll_order() static function for now.
+ <varlistentry> + <term><literal>ALTER COLLATION</literal></term> + <listitem> + <para> + This command declares that the index is compatible with the currently + installed version of the collations that determine its order. It is used + to silence warnings caused by collation version incompatibilities and + should be called after rebuilding the index or otherwise verifying its + consistency. Be aware that incorrect use of this command can hide index + corruption. + </para> + </listitem> + </varlistentry>This sounds like something that you need to do after you reindex, but
that's not true, is it? This is something you can do *instead* of
reindex, to make it shut up about versions. Shouldn't it be something
like "... should be issued only if the ordering is known not to have
changed since the index was built"?Indeed. We should also probably explicitly mention that if the
situation is unknown, REINDEX is the safe alternative to choose.
Done.
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION +UPDATE pg_depend SET refobjversion = 'not a version' +WHERE refclassid = 'pg_collation'::regclass +AND objid::regclass::text = 'icuidx17_part' +AND refobjversion IS NOT NULL; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + objid +--------------- + icuidx17_part +(1 row) + +ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION; +SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version'; + objid +------- +(0 rows) +Would it be better to put refobjversion = 'not a version' in the
SELECT list, instead of the WHERE clause, with a WHERE clause that
hits that one row, so that we can see that the row still exists after
the REFRESH VERSION (while still hiding the unstable version string)?Agreed, I'll change that.
And done.
An induced change in the attached v10 is that
pg_collation_actual_version() SQL function now also returns an empty
string if the collation version is unknown. That's not strictly
required, but it makes things more consistent and help writing queries
that finds which indexes may be broken. That function's behavior
wasn't documented for unknown version, so I also added some
clarification.
Attachments:
0003-Implement-type-regcollation-v10.patchapplication/octet-stream; name=0003-Implement-type-regcollation-v10.patchDownload
From 983493a3d22009693b10f54682bf0de6bd5cefda Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH 3/6] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
4 files changed, 185 insertions(+)
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index eb3c1a88d1..2b7964b835 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6663,6 +6663,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7449,6 +7458,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 4cf2b9df7b..e4d9406730 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
0001-Remove-pg_collation.collversion-v10.patchapplication/octet-stream; name=0001-Remove-pg_collation.collversion-v10.patchDownload
From 728c5f019c6a35ad4fb1e491dfc8a3071951a039 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/6] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
18 files changed, 11 insertions(+), 290 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index a10b66569b..f187a1af65 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index ceda48e0fc..44d82ca0b3 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21103,10 +21103,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 85f726ae06..493aa21a14 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 54ad62bb7f..bf4f793ba2 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3183,16 +3183,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5172,9 +5162,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 5b1ba143b1..4ec777a78c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3269,9 +3261,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 96e7fdbcfe..d1ce351200 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10324,21 +10323,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index bb85b5e52a..e57e05b9ee 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1798,10 +1798,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2948,10 +2944,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 25fb7e2ebf..597c1241f9 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d92a6626a5..6816d18ce3 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13500,7 +13500,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13572,7 +13576,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index df7d1d498c..9a9e145b4c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index da0706add5..c96d027362 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
0002-Add-pg_depend.refobjversion-v10.patchapplication/octet-stream; name=0002-Add-pg_depend.refobjversion-v10.patchDownload
From 1df4072b2eb67309b06c64ffadb6bd98c78b3904 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 28 ++++++++++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 3 ++
src/test/regress/expected/misc_sanity.out | 7 ++--
6 files changed, 66 insertions(+), 33 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index c4a4df25b8..78c31baa34 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2681,7 +2684,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f9af245eec..7fdbdf0ae8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 5302973379..887b37e3d0 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0cd6fcf027..77cf0612ed 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..41efb4a2c8 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
@@ -103,9 +103,10 @@ ORDER BY 1, 2;
pg_class | relacl | aclitem[]
pg_class | reloptions | text[]
pg_class | relpartbound | pg_node_tree
+ pg_depend | refobjversion | text
pg_index | indexprs | pg_node_tree
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
--
2.20.1
0004-Track-collation-versions-for-indexes-v10.patchapplication/octet-stream; name=0004-Track-collation-versions-for-indexes-v10.patchDownload
From 785675b6310cb342dfaa645674bc03456e78b879 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 4/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 6 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 56 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 1 +
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
24 files changed, 971 insertions(+), 67 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 44d82ca0b3..62b95cd52b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21103,7 +21103,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e31478bf91..46e9d74a97 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8880586c37..c6c51e0a48 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2628,6 +2785,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3605,6 +3773,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3d2b1cc911..0eaefbd032 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..7e087fefc4 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 8d7572da51..6e91518694 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 493aa21a14..58c237b2ee 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -289,11 +289,9 @@ pg_collation_actual_version(PG_FUNCTION_ARGS)
ReleaseSysCache(tp);
version = get_collation_actual_version(collprovider, collcollate);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 597c1241f9..a8bc5c363b 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -1462,6 +1464,10 @@ pg_newlocale_from_collation(Oid collid)
* A particular provider must always either return a non-NULL string or return
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
*/
char *
get_collation_actual_version(char collprovider, const char *collcollate)
@@ -1498,9 +1504,57 @@ get_collation_actual_version(char collprovider, const char *collcollate)
#endif
}
- return collversion;
+ if (!collversion)
+ return "";
+ else
+ return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..8750bfc36f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e1a5ab3df3..2bf6f868a5 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..aa06aca90a 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ddf3a63c3..92dcda5bdc 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
0005-Preserve-index-dependencies-on-collation-during-pg_u-v10.patchapplication/octet-stream; name=0005-Preserve-index-dependencies-on-collation-during-pg_u-v10.patchDownload
From a9a69c7cbce0d72161375d9b435288cfe29aad63 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/6] Preserve index dependencies on collation during
pg_upgrade
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for all or a single collation, for a specific
index.
Also teach pg_dump to call this function for all indexes, including indexes
created for constraints, when run with --binary-upgrade flag.
When pg_upgrade is run against an older version, collation versions are not
known and pg_dump will by default emit an alter index command to mark all
collation versions as unkown. However, it's possible that pg_upgrade is run
without upgrading the underlying collation libraries, so a new option
--collation-binary-compatible is added to avoid this behavior. This will
result in running pg_dump with a new --unknown-collations-binary-compatible
option, that can only be used in binary upgrade mode, to prevent pg_dump from
emitting call to this new function if the dependent collation version is
unknown. Note that if the collation version is known, this flag won't change
the behavior and the previous collation version will be preserved.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 512 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..b62a1d4eef 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ When upgrading from a PostgreSQL major version 12 or older, or from a
+ PostgreSQL major version that didn't have support for collation
+ versioning to a major version that now supports it, all indexes will be
+ marked as depending on an unknown collation version, as such versions
+ weren't tracked. As a result, numerous warning messages will be
+ emitted as it can be a sign of a corrupted index. If you're not
+ upgrading the collation libraries, and if you're absolutly certain that
+ all existing indexes are compatible with the current collation
+ libraries, you can use this flag to change this behavior and mark all
+ indexes as depending on current collation libraries version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c6c51e0a48..a5ac2dce5c 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3149,6 +3149,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 0d9e55cdcf..ea7c25ee35 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 6816d18ce3..c7bfdc2743 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -288,6 +289,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -388,6 +391,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -708,6 +712,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6856,7 +6864,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6892,7 +6902,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6917,7 +6982,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6956,7 +7023,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6991,7 +7060,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7022,7 +7093,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7056,7 +7129,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7096,6 +7171,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7121,6 +7198,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16389,10 +16468,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16452,6 +16532,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16481,6 +16565,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18438,6 +18537,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 21004e5078..6d1df24080 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 4a9764c2d2..216a704bb7 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -914,9 +933,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1178,6 +1198,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1203,6 +1224,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1238,6 +1260,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1260,6 +1283,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1281,6 +1305,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1302,6 +1327,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1665,6 +1691,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1679,7 +1706,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2347,6 +2374,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2540,6 +2568,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2607,6 +2636,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2678,6 +2708,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3145,6 +3176,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3160,6 +3192,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3292,16 +3325,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3325,6 +3395,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3375,16 +3449,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3432,6 +3519,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3485,79 +3578,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index eed70fff4f..d36d6a283f 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index b156b516cc..af34dff920 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -294,6 +294,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 8750bfc36f..664bf42956 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2b7964b835..abdb99a6ce 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10194,6 +10194,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
0006-Add-a-new-ALTER-INDEX-name-ALTER-COLLATION-name-REFR-v10.patchapplication/octet-stream; name=0006-Add-a-new-ALTER-INDEX-name-ALTER-COLLATION-name-REFR-v10.patchDownload
From 1625d485d9c0390c05d6370ce2bd5ba3e56e88d8 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH 6/6] Add a new ALTER INDEX name ALTER COLLATION name REFRESH
VERSION
This command allows privileged users to specifify that the currently installed
collation library version, for a specific collation, is binary compatible with
the one that was installed when the specified index was built for the last
time.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..512322b08a 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of the collations that determine its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you're not sure about the
+ collation ordering changes, using <xref linkend="sql-reindex"/> is a safe
+ alternative, but will consume more resources.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index a5ac2dce5c..540f869898 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3149,7 +3149,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b7c8d663fc..feaea2024e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -553,6 +554,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3871,6 +3873,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4038,6 +4044,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ...
+ * REFRESH VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4604,6 +4616,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17269,3 +17286,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index bf4f793ba2..57c87ded58 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3174,6 +3174,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d1ce351200..5bd6cc03d8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2569,6 +2569,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index dc03fbde13..4cc1646763 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -807,6 +808,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1697,7 +1712,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1743,6 +1758,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 69d163f3cc..065488374e 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index c96d027362..9f637597a7 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 09512c0f66..68f75ceaaf 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2027,6 +2027,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index c8f1a620d2..c52347fcde 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -800,6 +800,17 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
Re: Collation versioning
On Thu, Feb 27, 2020 at 3:29 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
[v10]
Thanks. I'll do some more testing and review soon. It'd be really
cool to get this into PG13.
FYI cfbot said:
+++ /home/travis/build/postgresql-cfbot/postgresql/src/test/regress/results/collate.icu.utf8.out
2020-02-26 14:45:52.114401999 +0000
...
- icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "default" | out of dateRe: Collation versioning
On Thu, Feb 27, 2020 at 04:10:14PM +1300, Thomas Munro wrote:
On Thu, Feb 27, 2020 at 3:29 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
[v10]
Thanks. I'll do some more testing and review soon. It'd be really
cool to get this into PG13.
Thanks!
FYI cfbot said:
+++ /home/travis/build/postgresql-cfbot/postgresql/src/test/regress/results/collate.icu.utf8.out 2020-02-26 14:45:52.114401999 +0000 ... - icuidx06_d_en_fr_ga | "default" | up to date + icuidx06_d_en_fr_ga | "default" | out of date
Oh. It turns out that pg_collation_actual_version() isn't handling the
default collation:
# SELECT pg_collation_actual_version(100), pg_collation_actual_version(c.oid)
FROM pg_database d
JOIN pg_collation c on c.collname = d.datcollate
WHERE datname = current_database();
pg_collation_actual_version | pg_collation_actual_version
-----------------------------+-----------------------------
<NULL> | 2.30
(1 row)
Fixed in v11 by changing pg_collation_actual_version() to handle default
collation too, as it seems a better behavior.
Attachments:
0001-Remove-pg_collation.collversion-v11.patchtext/x-diff; charset=us-asciiDownload
From 88805eabf495c98361bd38d180edf8a109990260 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/6] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
18 files changed, 11 insertions(+), 290 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index a10b66569b..f187a1af65 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 28035f1635..8fa131d60d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21104,10 +21104,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 85f726ae06..493aa21a14 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 54ad62bb7f..bf4f793ba2 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3183,16 +3183,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5172,9 +5162,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 5b1ba143b1..4ec777a78c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3269,9 +3261,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 96e7fdbcfe..d1ce351200 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10324,21 +10323,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index bb85b5e52a..e57e05b9ee 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1798,10 +1798,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2948,10 +2944,6 @@ CreateCommandTag(Node *parsetree)
tag = "DROP SUBSCRIPTION";
break;
- case T_AlterCollationStmt:
- tag = "ALTER COLLATION";
- break;
-
case T_PrepareStmt:
tag = "PREPARE";
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 25fb7e2ebf..597c1241f9 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1342,8 +1342,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1445,41 +1443,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d92a6626a5..6816d18ce3 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13500,7 +13500,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13572,7 +13576,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index df7d1d498c..9a9e145b4c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index da0706add5..c96d027362 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.25.1
0002-Add-pg_depend.refobjversion-v11.patchtext/x-diff; charset=us-asciiDownload
From ca05c47f0442d08c4adab364a6210d853255416a Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 28 ++++++++++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 3 ++
src/test/regress/expected/misc_sanity.out | 7 ++--
6 files changed, 66 insertions(+), 33 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index c4a4df25b8..78c31baa34 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2681,7 +2684,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f9af245eec..7fdbdf0ae8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 5302973379..887b37e3d0 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0cd6fcf027..77cf0612ed 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..41efb4a2c8 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
@@ -103,9 +103,10 @@ ORDER BY 1, 2;
pg_class | relacl | aclitem[]
pg_class | reloptions | text[]
pg_class | relpartbound | pg_node_tree
+ pg_depend | refobjversion | text
pg_index | indexprs | pg_node_tree
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
--
2.25.1
0003-Implement-type-regcollation-v11.patchtext/x-diff; charset=us-asciiDownload
From fa922804d1facffa6a74662120dd13f3f46f7d65 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH 3/6] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
4 files changed, 185 insertions(+)
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 07a86c7b7b..f8fb62c623 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6666,6 +6666,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7452,6 +7461,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 4cf2b9df7b..e4d9406730 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.25.1
0004-Track-collation-versions-for-indexes-v11.patchtext/x-diff; charset=us-asciiDownload
From 250b4f50f0c4fcb0362c68a050a6630013aa9bcb Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 4/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
24 files changed, 968 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8fa131d60d..f1ef11d60d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21104,7 +21104,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e31478bf91..46e9d74a97 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8880586c37..c6c51e0a48 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2628,6 +2785,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3605,6 +3773,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3d2b1cc911..0eaefbd032 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..7e087fefc4 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 8d7572da51..6e91518694 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 493aa21a14..05709fab77 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -272,28 +272,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 597c1241f9..7dacceefeb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -142,6 +144,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1463,7 +1468,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1501,6 +1506,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..8750bfc36f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e1a5ab3df3..2bf6f868a5 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ddf3a63c3..92dcda5bdc 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.25.1
0005-Preserve-index-dependencies-on-collation-during-pg_u-v11.patchtext/x-diff; charset=us-asciiDownload
From d3bb2d18e9f1c308baeb731a5d041a5476e13225 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/6] Preserve index dependencies on collation during
pg_upgrade
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for all or a single collation, for a specific
index.
Also teach pg_dump to call this function for all indexes, including indexes
created for constraints, when run with --binary-upgrade flag.
When pg_upgrade is run against an older version, collation versions are not
known and pg_dump will by default emit an alter index command to mark all
collation versions as unkown. However, it's possible that pg_upgrade is run
without upgrading the underlying collation libraries, so a new option
--collation-binary-compatible is added to avoid this behavior. This will
result in running pg_dump with a new --unknown-collations-binary-compatible
option, that can only be used in binary upgrade mode, to prevent pg_dump from
emitting call to this new function if the dependent collation version is
unknown. Note that if the collation version is known, this flag won't change
the behavior and the previous collation version will be preserved.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 512 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..b62a1d4eef 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ When upgrading from a PostgreSQL major version 12 or older, or from a
+ PostgreSQL major version that didn't have support for collation
+ versioning to a major version that now supports it, all indexes will be
+ marked as depending on an unknown collation version, as such versions
+ weren't tracked. As a result, numerous warning messages will be
+ emitted as it can be a sign of a corrupted index. If you're not
+ upgrading the collation libraries, and if you're absolutly certain that
+ all existing indexes are compatible with the current collation
+ libraries, you can use this flag to change this behavior and mark all
+ indexes as depending on current collation libraries version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c6c51e0a48..a5ac2dce5c 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3149,6 +3149,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 0d9e55cdcf..ea7c25ee35 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 6816d18ce3..c7bfdc2743 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -288,6 +289,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -388,6 +391,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -708,6 +712,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6856,7 +6864,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6892,7 +6902,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6917,7 +6982,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6956,7 +7023,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6991,7 +7060,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7022,7 +7093,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7056,7 +7129,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7096,6 +7171,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7121,6 +7198,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16389,10 +16468,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16452,6 +16532,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16481,6 +16565,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18438,6 +18537,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 21004e5078..6d1df24080 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index eed70fff4f..d36d6a283f 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index b156b516cc..af34dff920 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -294,6 +294,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 8750bfc36f..664bf42956 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8fb62c623..d201eb762b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10200,6 +10200,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.25.1
0006-Add-a-new-ALTER-INDEX-name-ALTER-COLLATION-name-REFR-v11.patchtext/x-diff; charset=us-asciiDownload
From b7a541b743161b0f5af1c7a97a8570db5ee5713c Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH 6/6] Add a new ALTER INDEX name ALTER COLLATION name REFRESH
VERSION
This command allows privileged users to specifify that the currently installed
collation library version, for a specific collation, is binary compatible with
the one that was installed when the specified index was built for the last
time.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..512322b08a 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of the collations that determine its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you're not sure about the
+ collation ordering changes, using <xref linkend="sql-reindex"/> is a safe
+ alternative, but will consume more resources.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index a5ac2dce5c..540f869898 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3149,7 +3149,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b7c8d663fc..feaea2024e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -553,6 +554,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3871,6 +3873,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4038,6 +4044,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ...
+ * REFRESH VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4604,6 +4616,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17269,3 +17286,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index bf4f793ba2..57c87ded58 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3174,6 +3174,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d1ce351200..5bd6cc03d8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2569,6 +2569,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b6b08d0ccb..dc8f82ab29 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -807,6 +808,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1697,7 +1712,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1743,6 +1758,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 69d163f3cc..065488374e 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index c96d027362..9f637597a7 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 09512c0f66..68f75ceaaf 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2027,6 +2027,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index c8f1a620d2..c52347fcde 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -800,6 +800,17 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.25.1
Re: Collation versioning
On Thu, Feb 27, 2020 at 08:45:35AM +0100, Julien Rouhaud wrote:
On Thu, Feb 27, 2020 at 04:10:14PM +1300, Thomas Munro wrote:
On Thu, Feb 27, 2020 at 3:29 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
[v10]
[...]
Fixed in v11 by changing pg_collation_actual_version() to handle default
collation too, as it seems a better behavior.
Rebased v12 due to conflict with the recent command tag commit (2f9661311b), no
other changes.
Attachments:
0001-Remove-pg_collation.collversion-v12.patchtext/plain; charset=us-asciiDownload
From 04e6a6641040e4eeb339d9937630cf7a8df091e2 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/6] Remove pg_collation.collversion.
A later patch will add version tracking for individual database objects
that depend on collations.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
18 files changed, 11 insertions(+), 290 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 34bc0d0526..c299501c7b 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 323366feb6..b2d991ac7f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..4241ec9f5a 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -88,72 +88,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 85f726ae06..493aa21a14 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e04c33e4ad..6a3b1b46e7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5173,9 +5163,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 5b1ba143b1..4ec777a78c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3269,9 +3261,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 96e7fdbcfe..d1ce351200 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10324,21 +10323,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 1b460a2612..21b04d5eb8 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1788,10 +1788,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2935,10 +2931,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3547,10 +3539,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ef1539044f..81d6f3819a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13501,7 +13501,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13573,7 +13577,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index df7d1d498c..9a9e145b4c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index da0706add5..c96d027362 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
0002-Add-pg_depend.refobjversion-v12.patchtext/plain; charset=us-asciiDownload
From 423963e5fcffacca4d712f68171a81d29b000be4 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions, for indexes, check constraints and so forth.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 28 ++++++++++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 3 ++
src/test/regress/expected/misc_sanity.out | 7 ++--
6 files changed, 66 insertions(+), 33 deletions(-)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index c4a4df25b8..78c31baa34 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2681,7 +2684,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f9af245eec..7fdbdf0ae8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0cd6fcf027..77cf0612ed 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..41efb4a2c8 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
@@ -103,9 +103,10 @@ ORDER BY 1, 2;
pg_class | relacl | aclitem[]
pg_class | reloptions | text[]
pg_class | relpartbound | pg_node_tree
+ pg_depend | refobjversion | text
pg_index | indexprs | pg_node_tree
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
--
2.20.1
0003-Implement-type-regcollation-v12.patchtext/plain; charset=us-asciiDownload
From 4cc2cbf3747723febac7f11511d14d44e023720a Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH 3/6] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
4 files changed, 185 insertions(+)
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 07a86c7b7b..f8fb62c623 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6666,6 +6666,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7452,6 +7461,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 4cf2b9df7b..e4d9406730 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
0004-Track-collation-versions-for-indexes-v12.patchtext/plain; charset=us-asciiDownload
From b301c9274bea743caabbc4f0296d148e1f97ee1b Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 4/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
24 files changed, 968 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b2d991ac7f..8414f2bfd9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,7 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e31478bf91..46e9d74a97 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1681f61727..9f9281a7af 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2634,6 +2791,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3611,6 +3779,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3d2b1cc911..0eaefbd032 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..7e087fefc4 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 8d7572da51..6e91518694 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 493aa21a14..05709fab77 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -272,28 +272,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..8750bfc36f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e1a5ab3df3..2bf6f868a5 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
0005-Preserve-index-dependencies-on-collation-during-pg_u-v12.patchtext/plain; charset=us-asciiDownload
From a2df7297a4300a7d0f2f396dc19e481680147efd Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/6] Preserve index dependencies on collation during
pg_upgrade
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for all or a single collation, for a specific
index.
Also teach pg_dump to call this function for all indexes, including indexes
created for constraints, when run with --binary-upgrade flag.
When pg_upgrade is run against an older version, collation versions are not
known and pg_dump will by default emit an alter index command to mark all
collation versions as unkown. However, it's possible that pg_upgrade is run
without upgrading the underlying collation libraries, so a new option
--collation-binary-compatible is added to avoid this behavior. This will
result in running pg_dump with a new --unknown-collations-binary-compatible
option, that can only be used in binary upgrade mode, to prevent pg_dump from
emitting call to this new function if the dependent collation version is
unknown. Note that if the collation version is known, this flag won't change
the behavior and the previous collation version will be preserved.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 512 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..b62a1d4eef 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ When upgrading from a PostgreSQL major version 12 or older, or from a
+ PostgreSQL major version that didn't have support for collation
+ versioning to a major version that now supports it, all indexes will be
+ marked as depending on an unknown collation version, as such versions
+ weren't tracked. As a result, numerous warning messages will be
+ emitted as it can be a sign of a corrupted index. If you're not
+ upgrading the collation libraries, and if you're absolutly certain that
+ all existing indexes are compatible with the current collation
+ libraries, you can use this flag to change this behavior and mark all
+ indexes as depending on current collation libraries version.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9f9281a7af..2729a9d203 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3155,6 +3155,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 0d9e55cdcf..ea7c25ee35 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 81d6f3819a..81cd24c391 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -288,6 +289,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -388,6 +391,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -708,6 +712,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6857,7 +6865,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6893,7 +6903,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6918,7 +6983,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6957,7 +7024,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6992,7 +7061,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7023,7 +7094,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7057,7 +7130,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7097,6 +7172,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7122,6 +7199,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16390,10 +16469,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16453,6 +16533,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16482,6 +16566,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18439,6 +18538,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 21004e5078..6d1df24080 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 4ef2036ecd..be2d137376 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 8750bfc36f..664bf42956 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8fb62c623..d201eb762b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10200,6 +10200,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
0006-Add-a-new-ALTER-INDEX-name-ALTER-COLLATION-name-REFR-v12.patchtext/plain; charset=us-asciiDownload
From 2b29d8ed74b45c5eeecc518ac44f7cda1a851c60 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH 6/6] Add a new ALTER INDEX name ALTER COLLATION name REFRESH
VERSION
This command allows privileged users to specifify that the currently installed
collation library version, for a specific collation, is binary compatible with
the one that was installed when the specified index was built for the last
time.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..512322b08a 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of the collations that determine its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you're not sure about the
+ collation ordering changes, using <xref linkend="sql-reindex"/> is a safe
+ alternative, but will consume more resources.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 2729a9d203..cf768879a5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3155,7 +3155,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 02a7c04fdb..f563bca0dc 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -553,6 +554,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3871,6 +3873,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4038,6 +4044,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ...
+ * REFRESH VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4604,6 +4616,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17220,3 +17237,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 6a3b1b46e7..03ca3bb698 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3175,6 +3175,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d1ce351200..5bd6cc03d8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2569,6 +2569,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b6b08d0ccb..dc8f82ab29 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -807,6 +808,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1697,7 +1712,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1743,6 +1758,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 69d163f3cc..065488374e 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index c96d027362..9f637597a7 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 09512c0f66..68f75ceaaf 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2027,6 +2027,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index c8f1a620d2..c52347fcde 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -800,6 +800,17 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
Re: Collation versioning
On Wed, Mar 4, 2020 at 10:01 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Thu, Feb 27, 2020 at 08:45:35AM +0100, Julien Rouhaud wrote:
On Thu, Feb 27, 2020 at 04:10:14PM +1300, Thomas Munro wrote:
On Thu, Feb 27, 2020 at 3:29 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
[v10]
[...]
Fixed in v11 by changing pg_collation_actual_version() to handle default
collation too, as it seems a better behavior.Rebased v12 due to conflict with the recent command tag commit (2f9661311b), no
other changes.
I'm still reviewing and testing the code, but here's a new patch set
with a bunch of small changes to documentation, tests and commit
messages:
0001-Remove-pg_collation.collversion-v13.patch
* removed ALTER COLLATION ... REFRESH VERSION from the docs
* removed ALTER COLLATION ... REFRESH VERSION from collate.linux.utf8.sql
(oops, that wasn't running on my FreeBSD system, or on cfbot's Ubuntu
image because it doesn't have "locales-all" installed; I will make a
note to fix that on cfbot)
0002-Add-pg_depend.refobjversion-v13.patch
* added new column to the documentation of the pg_depend catalog
0003-Implement-type-regcollation-v13.patch
* added regcollation to a list of such types in datatype.sgml
0004-Track-collation-versions-for-indexes-v13.patch
* added a note to the documentation's list of reasons why REINDEX
might be needed
0005-Preserve-index-dependencies-on-collation-during--v13.patch
* minor rewording of the docs, I hope you like it
0006-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS-v13.patch
* minor rewording of the docs, I hope you like it
0007-doc-Add-Collation-Versions-section-v13.patch
* new: a brief introduction to this topic
Attachments:
0001-Remove-pg_collation.collversion-v13.patchtext/x-patch; charset=US-ASCII; name=0001-Remove-pg_collation.collversion-v13.patchDownload
From e2fe5c94667adbdb8e7ebc3806fe9c70efb69e2a Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH 1/7] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 323366feb6..b2d991ac7f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 85f726ae06..493aa21a14 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -67,7 +67,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -165,9 +164,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -214,9 +210,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -225,7 +218,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -276,80 +268,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -607,7 +525,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -668,7 +585,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -730,7 +646,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a12c8d011b..f4be6cad4c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13497,7 +13497,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13569,7 +13573,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index df7d1d498c..9a9e145b4c 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
0002-Add-pg_depend.refobjversion-v13.patchtext/x-patch; charset=US-ASCII; name=0002-Add-pg_depend.refobjversion-v13.patchDownload
From a05a62e90612c9390205f763f57721e9596f8d6c Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH 2/7] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 13 +++++++
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 28 ++++++++++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 3 ++
src/test/regress/expected/misc_sanity.out | 7 ++--
7 files changed, 79 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c2d33c76e0..ad44c44530 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2950,6 +2950,15 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</entry>
</row>
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object; see text
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3116,6 +3125,10 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
must be satisfied.
</para>
+ <para>
+ The only current use of <structfield>refobjversion</structfield> is to
+ record dependencies between indexes and collation versions.
+ </para>
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index c4a4df25b8..78c31baa34 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1602,7 +1602,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1689,7 +1690,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1709,7 +1711,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2681,7 +2684,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f9af245eec..7fdbdf0ae8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0cd6fcf027..77cf0612ed 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..41efb4a2c8 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
@@ -103,9 +103,10 @@ ORDER BY 1, 2;
pg_class | relacl | aclitem[]
pg_class | reloptions | text[]
pg_class | relpartbound | pg_node_tree
+ pg_depend | refobjversion | text
pg_index | indexprs | pg_node_tree
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
--
2.20.1
0003-Implement-type-regcollation-v13.patchtext/x-patch; charset=US-ASCII; name=0003-Implement-type-regcollation-v13.patchDownload
From b886c59e683d507c5421f7a46cfebccdbea3005c Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH 3/7] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 4 +
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
5 files changed, 189 insertions(+)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index 410eaedcb7..9a86d645a0 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4496,6 +4496,10 @@ INSERT INTO mytable VALUES(-1); -- fails
<primary>regtype</primary>
</indexterm>
+ <indexterm zone="datatype-oid">
+ <primary>regcollation</primary>
+ </indexterm>
+
<indexterm zone="datatype-oid">
<primary>regconfig</primary>
</indexterm>
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7fb574f9dc..c7c25e1eae 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6666,6 +6666,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7446,6 +7455,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b00597d6ff..ad777e37c6 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
0004-Track-collation-versions-for-indexes-v13.patchtext/x-patch; charset=US-ASCII; name=0004-Track-collation-versions-for-indexes-v13.patchDownload
From 9581ecb0319921b9f23791bf2eee578535fbb265 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH 4/7] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
25 files changed, 974 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b2d991ac7f..8414f2bfd9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,7 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..4546789446 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7223679033..7a1033bf79 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3612,6 +3780,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..7e087fefc4 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..fe14b6bd38 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 493aa21a14..05709fab77 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -272,28 +272,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..8750bfc36f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
0005-Preserve-index-dependencies-on-collation-during--v13.patchtext/x-patch; charset=US-ASCII; name=0005-Preserve-index-dependencies-on-collation-during--v13.patchDownload
From ff0738ce99f259bb4ff6d0bb2a4749f3a89d99bd Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH 5/7] Preserve index dependencies on collation during
pg_upgrade.
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for collations. pg_dump will call this
function to preserve the version information for all indexes, if run with the
--binary-upgrade option.
When pg_upgrade is used to upgrade from a version of PostgreSQL that didn't
support per-object versions, the version will be recorded as unknown. If you
believe that the collation definitions haven't changed since indexes were last
built, you can use --collation-binary-compatible to tell pg_upgrade to assume
that the current versions are compatible with the existing indexes, and it will
pass --unknown-collations-binary-compatible through to pg_dump and mark them as
having the current version (according to the collation provider).
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 512 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..9794fc5234 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7a1033bf79..a1732764e4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..66511b0a45 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f4be6cad4c..3c569e02b4 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +388,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -704,6 +708,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6853,7 +6861,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6889,7 +6899,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6914,7 +6979,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6953,7 +7020,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6988,7 +7057,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7019,7 +7090,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7053,7 +7126,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7093,6 +7168,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7118,6 +7195,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16386,10 +16465,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16449,6 +16529,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16478,6 +16562,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18431,6 +18530,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 852020b66c..31cc4de4d8 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 4ef2036ecd..be2d137376 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 8750bfc36f..664bf42956 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c7c25e1eae..e403107946 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10194,6 +10194,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
0006-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS-v13.patchtext/x-patch; charset=US-ASCII; name=0006-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS-v13.patchDownload
From 8a7fb479a15ed325013df52a29531cf988e709ed Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH 6/7] Add ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..744789b1bb 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index a1732764e4..51621ea1c2 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,7 +3156,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3eb861bfbf..92b2d9370a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -553,6 +554,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3871,6 +3873,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4038,6 +4044,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ...
+ * REFRESH VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4604,6 +4616,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17218,3 +17235,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7caf0f2f53..6c0a6a2732 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3175,6 +3175,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 804cbafda4..89fe0b38f8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2570,6 +2570,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 174c3db623..0929003b8e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1705,7 +1720,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1751,6 +1766,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 69d163f3cc..065488374e 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 079fe1a5f3..64b3e40b70 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 09512c0f66..68f75ceaaf 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2027,6 +2027,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index c8f1a620d2..c52347fcde 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -800,6 +800,17 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
0007-doc-Add-Collation-Versions-section-v13.patchtext/x-patch; charset=US-ASCII; name=0007-doc-Add-Collation-Versions-section-v13.patchDownload
From e45a1266a276665943417007c31770bd338ac995 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH 7/7] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 20cdfabd7b..d4aa300c5a 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -936,6 +936,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</para>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Wed, Mar 11, 2020 at 04:13:17PM +1300, Thomas Munro wrote:
On Wed, Mar 4, 2020 at 10:01 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Thu, Feb 27, 2020 at 08:45:35AM +0100, Julien Rouhaud wrote:
On Thu, Feb 27, 2020 at 04:10:14PM +1300, Thomas Munro wrote:
On Thu, Feb 27, 2020 at 3:29 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
[v10]
[...]
Fixed in v11 by changing pg_collation_actual_version() to handle default
collation too, as it seems a better behavior.Rebased v12 due to conflict with the recent command tag commit (2f9661311b), no
other changes.I'm still reviewing and testing the code, but here's a new patch set
with a bunch of small changes to documentation, tests and commit
messages:
Thanks a lot Thomas! FTR one of the patch has a minor conflict due to
3c173a53a82, so I'm attaching a v14 to make cfbot happy and also benefit from
windows coverage just in case. I also tried to add some missing reviewers to
the patches, I hope I didn't forget anyone.
0001-Remove-pg_collation.collversion-v13.patch
* removed ALTER COLLATION ... REFRESH VERSION from collate.linux.utf8.sql
(oops, that wasn't running on my FreeBSD system, or on cfbot's Ubuntu
image because it doesn't have "locales-all" installed; I will make a
note to fix that on cfbot)
Wow I don't know how that happened since I did remove that in v2 ([1]/messages/by-id/CAOBaU_aL=4f67L+m2s28DmiaacZ=Dn75BZY-HGmEq1WquGa-Jg@mail.gmail.com). Thanks!
0002-Add-pg_depend.refobjversion-v13.patch
* added new column to the documentation of the pg_depend catalog0003-Implement-type-regcollation-v13.patch
* added regcollation to a list of such types in datatype.sgml0004-Track-collation-versions-for-indexes-v13.patch
* added a note to the documentation's list of reasons why REINDEX
might be needed0005-Preserve-index-dependencies-on-collation-during--v13.patch
* minor rewording of the docs, I hope you like it0006-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS-v13.patch
* minor rewording of the docs, I hope you like it0007-doc-Add-Collation-Versions-section-v13.patch
* new: a brief introduction to this topic
I'm quite happy with all the changes, and especially the new section about
collation version in 0007.
Minor gripe in pgugprade.sgml, as Peter E. mentioned in [2]/messages/by-id/3ede40e7-5799-7096-dc5f-d7beda8e7145@2ndquadrant.com the documentation
shouldn't focus on pg13- / pg13+ upgrade only. Or should we keep that for when
additional system will be supported post pg13?
[1]: /messages/by-id/CAOBaU_aL=4f67L+m2s28DmiaacZ=Dn75BZY-HGmEq1WquGa-Jg@mail.gmail.com
[2]: /messages/by-id/3ede40e7-5799-7096-dc5f-d7beda8e7145@2ndquadrant.com
Attachments:
v14-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 9b4102c2f0dada21c8970b6a0450e49c73095c97 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v14 1/7] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 323366feb6..b2d991ac7f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ad039e97a5..febe582d27 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13501,7 +13501,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13573,7 +13577,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v14-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From fa89bd9bc5e8c532689e998d826bdabd2a40c3ea Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v14 2/7] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 13 +++++++
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 28 ++++++++++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 3 ++
src/test/regress/expected/misc_sanity.out | 7 ++--
7 files changed, 79 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c2d33c76e0..ad44c44530 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2950,6 +2950,15 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</entry>
</row>
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object; see text
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3116,6 +3125,10 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
must be satisfied.
</para>
+ <para>
+ The only current use of <structfield>refobjversion</structfield> is to
+ record dependencies between indexes and collation versions.
+ </para>
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index ffd52c1153..660033b9c1 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1603,7 +1603,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1690,7 +1691,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1710,7 +1712,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2682,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index f9af245eec..7fdbdf0ae8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0cd6fcf027..77cf0612ed 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..41efb4a2c8 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
@@ -103,9 +103,10 @@ ORDER BY 1, 2;
pg_class | relacl | aclitem[]
pg_class | reloptions | text[]
pg_class | relpartbound | pg_node_tree
+ pg_depend | refobjversion | text
pg_index | indexprs | pg_node_tree
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
--
2.20.1
v14-0003-Implement-type-regcollation.patchtext/plain; charset=us-asciiDownload
From ffeb0292591816c77949efe267c2db4f8c210ea7 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH v14 3/7] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 4 +
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
5 files changed, 189 insertions(+)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index 410eaedcb7..9a86d645a0 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4496,6 +4496,10 @@ INSERT INTO mytable VALUES(-1); -- fails
<primary>regtype</primary>
</indexterm>
+ <indexterm zone="datatype-oid">
+ <primary>regcollation</primary>
+ </indexterm>
+
<indexterm zone="datatype-oid">
<primary>regconfig</primary>
</indexterm>
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7fb574f9dc..c7c25e1eae 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6666,6 +6666,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7446,6 +7455,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b00597d6ff..ad777e37c6 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
v14-0004-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From c02084df194d110ac2bb2c2ef59f7511e0a59f19 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v14 4/7] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
25 files changed, 974 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b2d991ac7f..8414f2bfd9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,7 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 660033b9c1..c0e36ef4cd 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1669,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1603,9 +1685,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1632,12 +1715,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,9 +1780,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1712,9 +1802,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1827,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1866,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1930,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2026,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2115,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2147,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2161,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2175,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2265,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2411,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2435,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2833,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..4546789446 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..369b5d43f1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3791,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..7e087fefc4 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..fe14b6bd38 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..8750bfc36f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v14-0005-Preserve-index-dependencies-on-collation-during-.patchtext/plain; charset=us-asciiDownload
From 4aa0941caaa42186c481cbf06f18bd52548f1575 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH v14 5/7] Preserve index dependencies on collation during
pg_upgrade.
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for collations. pg_dump will call this
function to preserve the version information for all indexes, if run with the
--binary-upgrade option.
When pg_upgrade is used to upgrade from a version of PostgreSQL that didn't
support per-object versions, the version will be recorded as unknown. If you
believe that the collation definitions haven't changed since indexes were last
built, you can use --collation-binary-compatible to tell pg_upgrade to assume
that the current versions are compatible with the existing indexes, and it will
pass --unknown-collations-binary-compatible through to pg_dump and mark them as
having the current version (according to the collation provider).
Author: Julien Rouhaud
Reviewed-by: Peter Eisentraut, Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 512 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..9794fc5234 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 369b5d43f1..68898636f5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..66511b0a45 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index febe582d27..df86a07fdd 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +388,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -704,6 +708,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6857,7 +6865,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6893,7 +6903,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6918,7 +6983,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -6957,7 +7024,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -6992,7 +7061,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7023,7 +7094,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7057,7 +7130,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7097,6 +7172,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7122,6 +7199,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16390,10 +16469,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16453,6 +16533,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
}
}
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16482,6 +16566,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18435,6 +18534,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 852020b66c..31cc4de4d8 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -364,6 +364,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 4ef2036ecd..be2d137376 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 8750bfc36f..664bf42956 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c7c25e1eae..e403107946 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10194,6 +10194,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
v14-0006-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From 2eccffb5d3a4f0603b823094f09d6f8c56fb1557 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v14 6/7] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..744789b1bb 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 68898636f5..e075e9927d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,7 +3156,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3eb861bfbf..92b2d9370a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -553,6 +554,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3871,6 +3873,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4038,6 +4044,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ...
+ * REFRESH VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4604,6 +4616,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17218,3 +17235,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7caf0f2f53..6c0a6a2732 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3175,6 +3175,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 804cbafda4..89fe0b38f8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2570,6 +2570,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 174c3db623..0929003b8e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1705,7 +1720,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1751,6 +1766,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 69d163f3cc..065488374e 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 079fe1a5f3..64b3e40b70 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 09512c0f66..68f75ceaaf 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2027,6 +2027,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index c8f1a620d2..c52347fcde 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -800,6 +800,17 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v14-0007-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 500f7d7f2c1170cd3a8cd469c78f5f71548afb27 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v14 7/7] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 20cdfabd7b..d4aa300c5a 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -936,6 +936,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</para>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Wed, Mar 11, 2020 at 09:44:32AM +0100, Julien Rouhaud wrote:
Thanks a lot Thomas! FTR one of the patch has a minor conflict due to
3c173a53a82, so I'm attaching a v14 to make cfbot happy and also benefit from
windows coverage just in case.
And v15 due to conflict with b08dee24a5 (Add pg_dump support for ALTER obj
DEPENDS ON EXTENSION).
Attachments:
v15-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 23e4c71d118db8ced68d1e7890b4da1c649dff06 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v15 1/7] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 323366feb6..b2d991ac7f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 28312e14ef..ce31d16bd1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13556,7 +13556,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13628,7 +13632,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v15-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From 5d0a90f51ccd9abb79adb744a8c0ad9a81cf33b6 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v15 2/7] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 13 +++++++
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 28 ++++++++++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 3 ++
src/test/regress/expected/misc_sanity.out | 7 ++--
7 files changed, 79 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c2d33c76e0..ad44c44530 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2950,6 +2950,15 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</entry>
</row>
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object; see text
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3116,6 +3125,10 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
must be satisfied.
</para>
+ <para>
+ The only current use of <structfield>refobjversion</structfield> is to
+ record dependencies between indexes and collation versions.
+ </para>
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index ffd52c1153..660033b9c1 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1603,7 +1603,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1690,7 +1691,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1710,7 +1712,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2682,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 596dafe19c..b7beb2884e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ab5e92bdc6..7d9fc866b9 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..41efb4a2c8 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
@@ -103,9 +103,10 @@ ORDER BY 1, 2;
pg_class | relacl | aclitem[]
pg_class | reloptions | text[]
pg_class | relpartbound | pg_node_tree
+ pg_depend | refobjversion | text
pg_index | indexprs | pg_node_tree
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
--
2.20.1
v15-0003-Implement-type-regcollation.patchtext/plain; charset=us-asciiDownload
From 6c3553dcc0df106fc6a41d0fcf8bd7183fe4246c Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH v15 3/7] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 4 +
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 ++++
src/include/catalog/pg_type.dat | 4 +
5 files changed, 189 insertions(+)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index 410eaedcb7..9a86d645a0 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4496,6 +4496,10 @@ INSERT INTO mytable VALUES(-1); -- fails
<primary>regtype</primary>
</indexterm>
+ <indexterm zone="datatype-oid">
+ <primary>regcollation</primary>
+ </indexterm>
+
<indexterm zone="datatype-oid">
<primary>regconfig</primary>
</indexterm>
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7fb574f9dc..c7c25e1eae 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6666,6 +6666,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7446,6 +7455,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b00597d6ff..ad777e37c6 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
--
2.20.1
v15-0004-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From b286b15c13c3660068736a1f3dc92b33d9b69eb7 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v15 4/7] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
25 files changed, 974 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b2d991ac7f..8414f2bfd9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,7 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 660033b9c1..c0e36ef4cd 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1669,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1603,9 +1685,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1632,12 +1715,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,9 +1780,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1712,9 +1802,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1827,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1866,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1930,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2026,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2115,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2147,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2161,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2175,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2265,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2411,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2435,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2833,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..4546789446 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..369b5d43f1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3791,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index b7beb2884e..3f113499b2 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..fe14b6bd38 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 7d9fc866b9..8858ceb3df 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v15-0005-Preserve-index-dependencies-on-collation-during-.patchtext/plain; charset=us-asciiDownload
From 866e0c7a3a7736154112fc1aa8dfbb48aa2cbda5 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH v15 5/7] Preserve index dependencies on collation during
pg_upgrade.
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for collations. pg_dump will call this
function to preserve the version information for all indexes, if run with the
--binary-upgrade option.
When pg_upgrade is used to upgrade from a version of PostgreSQL that didn't
support per-object versions, the version will be recorded as unknown. If you
believe that the collation definitions haven't changed since indexes were last
built, you can use --collation-binary-compatible to tell pg_upgrade to assume
that the current versions are compatible with the existing indexes, and it will
pass --unknown-collations-binary-compatible through to pg_dump and mark them as
having the current version (according to the collation provider).
Author: Julien Rouhaud
Reviewed-by: Peter Eisentraut, Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 512 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..9794fc5234 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 369b5d43f1..68898636f5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..66511b0a45 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ce31d16bd1..e3278369a2 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +388,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -704,6 +708,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6906,7 +6914,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6942,7 +6952,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6967,7 +7032,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7006,7 +7073,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7041,7 +7110,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7072,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7106,7 +7179,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7146,6 +7221,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7171,6 +7248,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16455,10 +16534,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16522,6 +16602,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16550,6 +16634,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18526,6 +18625,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0c6444ef6..e3369451a7 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -365,6 +365,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 4ef2036ecd..be2d137376 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 8858ceb3df..5bbf59ea2f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c7c25e1eae..e403107946 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10194,6 +10194,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
v15-0006-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From 915cd1973f44abda58079c2b3f9264f80aaea532 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v15 6/7] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..744789b1bb 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 68898636f5..e075e9927d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,7 +3156,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3eb861bfbf..92b2d9370a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -553,6 +554,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3871,6 +3873,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4038,6 +4044,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ...
+ * REFRESH VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4604,6 +4616,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17218,3 +17235,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7caf0f2f53..6c0a6a2732 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3175,6 +3175,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 804cbafda4..89fe0b38f8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2570,6 +2570,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 174c3db623..0929003b8e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1705,7 +1720,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1751,6 +1766,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 69d163f3cc..065488374e 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 079fe1a5f3..64b3e40b70 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 09512c0f66..68f75ceaaf 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2027,6 +2027,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index c8f1a620d2..c52347fcde 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -800,6 +800,17 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v15-0007-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 2cc9f8d9788d6051028a639530cc33a5c2a645bb Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v15 7/7] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 20cdfabd7b..d4aa300c5a 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -936,6 +936,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</para>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Thu, Mar 12, 2020 at 03:00:26PM +0100, Julien Rouhaud wrote:
And v15 due to conflict with b08dee24a5 (Add pg_dump support for ALTER obj
DEPENDS ON EXTENSION).
I have looked at patches 0001~0003 in the set for now. 0001 looks
clean to me.
In patch 0002, you have the following addition:
@@ -103,9 +103,10 @@ ORDER BY 1, 2;
pg_class | relacl | aclitem[]
pg_class | reloptions | text[]
pg_class | relpartbound | pg_node_tree
+ pg_depend | refobjversion | text
This comes from a regression test doing a sanity check to look for
catalogs which have a toastable column but no toast tables. As an
exception, it should be documented in the test's comment. Actually,
does it need to be an exception? This does not depend on
relation-level facilities so there should be no risk of recursive
dependencies, though I have not looked in details at this part.+ <para>
+ The only current use of <structfield>refobjversion</structfield> is to
+ record dependencies between indexes and collation versions.
+ </para>
[...]
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object; see text
+ </entry>
+ </row>
Couldn't you merge both paragraphs here?Regarding patch 0003, it would be nice to include some tests
independent on the rest and making use of the new functions. These
normally go in regproc.sql. For example with a collation that needs
double quotes as this is not obvious:
=# select regcollation('"POSIX"');
regcollation
--------------
"POSIX"
(1 row)
On top of that, this needs tests with to_regcollation() and tests with
schema-qualified collations.
Documentation for to_regcollation() is missing. And it looks that
many parts of the documentation are missing an update. One example in
datatype.sgml:
Type <type>oid</type> represents an object identifier. There are also
several alias types for <type>oid</type>: <type>regproc</type>,
<type>regprocedure</type>, <type>regoper</type>, <type>regoperator</type>,
<type>regclass</type>, <type>regtype</type>, <type>regrole</type>,
<type>regnamespace</type>, <type>regconfig</type>, and
<type>regdictionary</type>. <xref linkend="datatype-oid-table"/> shows an
overview.
At quick glance, there are more sections in need of a refresh..
--
Michael
Re: Collation versioning
On Mon, Mar 16, 2020 at 04:57:38PM +0900, Michael Paquier wrote:
On Thu, Mar 12, 2020 at 03:00:26PM +0100, Julien Rouhaud wrote:
And v15 due to conflict with b08dee24a5 (Add pg_dump support for ALTER obj
DEPENDS ON EXTENSION).I have looked at patches 0001~0003 in the set for now.
Thanks!
In patch 0002, you have the following addition: @@ -103,9 +103,10 @@ ORDER BY 1, 2; pg_class | relacl | aclitem[] pg_class | reloptions | text[] pg_class | relpartbound | pg_node_tree + pg_depend | refobjversion | text This comes from a regression test doing a sanity check to look for catalogs which have a toastable column but no toast tables. As an exception, it should be documented in the test's comment. Actually, does it need to be an exception? This does not depend on relation-level facilities so there should be no risk of recursive dependencies, though I have not looked in details at this part.
I totally missed that, and I agree that there's no need for an exception, so
fixed.
+ <para> + The only current use of <structfield>refobjversion</structfield> is to + record dependencies between indexes and collation versions. + </para> [...] + <row> + <entry><structfield>refobjversion</structfield></entry> + <entry><type>text</type></entry> + <entry></entry> + <entry> + An optional version for the referenced object; see text + </entry> + </row> Couldn't you merge both paragraphs here?
Done.
Regarding patch 0003, it would be nice to include some tests
independent on the rest and making use of the new functions. These
normally go in regproc.sql. For example with a collation that needs
double quotes as this is not obvious:
=# select regcollation('"POSIX"');
regcollation
--------------
"POSIX"
(1 row)On top of that, this needs tests with to_regcollation() and tests with
schema-qualified collations.
Done too, using the same collation name, for both with and without schema
qualification.
Documentation for to_regcollation() is missing. And it looks that
many parts of the documentation are missing an update. One example in
datatype.sgml:
Type <type>oid</type> represents an object identifier. There are also
several alias types for <type>oid</type>: <type>regproc</type>,
<type>regprocedure</type>, <type>regoper</type>, <type>regoperator</type>,
<type>regclass</type>, <type>regtype</type>, <type>regrole</type>,
<type>regnamespace</type>, <type>regconfig</type>, and
<type>regdictionary</type>. <xref linkend="datatype-oid-table"/> shows an
overview.
At quick glance, there are more sections in need of a refresh..
Indeed. I found missing reference in datatype.sgml; func.sgml and
pgupgrade.sgml.
v16 attached.
Attachments:
v16-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 38c13420058275471e115e98cc54ce6600e16ed9 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v16 1/7] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 323366feb6..b2d991ac7f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 28312e14ef..ce31d16bd1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13556,7 +13556,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13628,7 +13632,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v16-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From c955a06ef24ecb46b5d8dbacaa5618f180a4a770 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v16 2/7] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 ++++++-
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 28 ++++++++++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 6 ++++
src/include/catalog/pg_depend.h | 3 ++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 76 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c2d33c76e0..4dacebf8c9 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2950,6 +2950,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</entry>
</row>
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3115,7 +3126,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index ffd52c1153..660033b9c1 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1603,7 +1603,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1690,7 +1691,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1710,7 +1712,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2682,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 596dafe19c..b7beb2884e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -44,7 +44,22 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+}
+
+/*
+ * As recordDependencyOn(), but also capture a version string so that changes
+ * in the referenced object can be detected. The meaning of the version
+ * string depends on the referenced object. Currently it is used for
+ * detecting changes in collation versions.
+ */
+void
+recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior)
+{
+ recordMultipleDependencies(depender, referenced, version, 1, behavior);
}
/*
@@ -54,6 +69,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +95,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +108,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +118,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ab5e92bdc6..7d9fc866b9 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -182,8 +182,14 @@ extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ const NameData *version,
+ DependencyType behavior);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const NameData *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v16-0003-Implement-type-regcollation.patchtext/plain; charset=us-asciiDownload
From 3f80a72bdffc50329f5699bc31f59be38fc8baac Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH v16 3/7] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by: Thomas Munro and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 23 +++-
doc/src/sgml/func.sgml | 38 ++++---
doc/src/sgml/ref/pgupgrade.sgml | 4 +-
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 +++
src/include/catalog/pg_type.dat | 4 +
src/test/regress/expected/regproc.out | 24 ++++
src/test/regress/sql/regproc.sql | 4 +
9 files changed, 256 insertions(+), 22 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index 410eaedcb7..86d16e7b13 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4496,6 +4496,10 @@ INSERT INTO mytable VALUES(-1); -- fails
<primary>regtype</primary>
</indexterm>
+ <indexterm zone="datatype-oid">
+ <primary>regcollation</primary>
+ </indexterm>
+
<indexterm zone="datatype-oid">
<primary>regconfig</primary>
</indexterm>
@@ -4522,12 +4526,12 @@ INSERT INTO mytable VALUES(-1); -- fails
system tables.
Type <type>oid</type> represents an object identifier. There are also
- several alias types for <type>oid</type>: <type>regproc</type>,
- <type>regprocedure</type>, <type>regoper</type>, <type>regoperator</type>,
- <type>regclass</type>, <type>regtype</type>, <type>regrole</type>,
- <type>regnamespace</type>, <type>regconfig</type>, and
- <type>regdictionary</type>. <xref linkend="datatype-oid-table"/> shows an
- overview.
+ several alias types for <type>oid</type>: <type>regcollation</type>,
+ <type>regproc</type>, <type>regprocedure</type>, <type>regoper</type>,
+ <type>regoperator</type>, <type>regclass</type>, <type>regtype</type>,
+ <type>regrole</type>, <type>regnamespace</type>, <type>regconfig</type>,
+ and <type>regdictionary</type>. <xref linkend="datatype-oid-table"/> shows
+ an overview.
</para>
<para>
@@ -4591,6 +4595,13 @@ SELECT * FROM pg_attribute
<entry><literal>564182</literal></entry>
</row>
+ <row>
+ <entry><type>regcollation</type></entry>
+ <entry><structname>pg_collation</structname></entry>
+ <entry>collation name</entry>
+ <entry><literal>"POSIX"</literal></entry>
+ </row>
+
<row>
<entry><type>regproc</type></entry>
<entry><structname>pg_proc</structname></entry>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b2d991ac7f..02779ab713 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -18177,6 +18177,10 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
<primary>to_regclass</primary>
</indexterm>
+ <indexterm>
+ <primary>to_regcollation</primary>
+ </indexterm>
+
<indexterm>
<primary>to_regproc</primary>
</indexterm>
@@ -18389,6 +18393,11 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
<entry><type>regclass</type></entry>
<entry>get the OID of the named relation</entry>
</row>
+ <row>
+ <entry><literal><function>to_regcollation(<parameter>coll_name</parameter>)</function></literal></entry>
+ <entry><type>regcollation</type></entry>
+ <entry>get the OID of the named collation</entry>
+ </row>
<row>
<entry><literal><function>to_regproc(<parameter>func_name</parameter>)</function></literal></entry>
<entry><type>regproc</type></entry>
@@ -18720,20 +18729,21 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
- The <function>to_regclass</function>, <function>to_regproc</function>,
- <function>to_regprocedure</function>, <function>to_regoper</function>,
- <function>to_regoperator</function>, <function>to_regtype</function>,
- <function>to_regnamespace</function>, and <function>to_regrole</function>
- functions translate relation, function, operator, type, schema, and role
- names (given as <type>text</type>) to objects of
- type <type>regclass</type>, <type>regproc</type>, <type>regprocedure</type>,
+ The <function>to_regclass</function>, <function>to_regcollation</function>,
+ <function>to_regproc</function>, <function>to_regprocedure</function>,
+ <function>to_regoper</function>, <function>to_regoperator</function>,
+ <function>to_regtype</function>, <function>to_regnamespace</function>, and
+ <function>to_regrole</function> functions translate relation, collation,
+ function, operator, type, schema, and role names (given as
+ <type>text</type>) to objects of type <type>regclass</type>,
+ <type>regcollation</type>, <type>regproc</type>, <type>regprocedure</type>,
<type>regoper</type>, <type>regoperator</type>, <type>regtype</type>,
- <type>regnamespace</type>, and <type>regrole</type>
- respectively. These functions differ from a cast from
- text in that they don't accept a numeric OID, and that they return null
- rather than throwing an error if the name is not found (or, for
- <function>to_regproc</function> and <function>to_regoper</function>, if
- the given name matches multiple objects).
+ <type>regnamespace</type>, and <type>regrole</type> respectively. These
+ functions differ from a cast from text in that they don't accept a numeric
+ OID, and that they return null rather than throwing an error if the name is
+ not found (or, for <function>to_regproc</function> and
+ <function>to_regoper</function>, if the given name matches multiple
+ objects).
</para>
<indexterm>
@@ -21095,7 +21105,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<row>
<entry>
<indexterm><primary>pg_collation_actual_version</primary></indexterm>
- <literal><function>pg_collation_actual_version(<type>oid</type>)</function></literal>
+ <literal><function>pg_collation_actual_version(<type>regcollation</type>)</function></literal>
</entry>
<entry><type>text</type></entry>
<entry>Return actual version of collation from operating system</entry>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..8e229135c2 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -773,8 +773,8 @@ psql --username=postgres --file=script.sql postgres
<para>
<application>pg_upgrade</application> does not support upgrading of databases
containing table columns using these <type>reg*</type> OID-referencing system data types:
- <type>regproc</type>, <type>regprocedure</type>, <type>regoper</type>,
- <type>regoperator</type>, <type>regconfig</type>, and
+ <type>regcollation</type>, <type>regproc</type>, <type>regprocedure</type>,
+ <type>regoper</type>, <type>regoperator</type>, <type>regconfig</type>, and
<type>regdictionary</type>. (<type>regtype</type> can be upgraded.)
</para>
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7fb574f9dc..c7c25e1eae 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6666,6 +6666,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7446,6 +7455,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b00597d6ff..ad777e37c6 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
diff --git a/src/test/regress/expected/regproc.out b/src/test/regress/expected/regproc.out
index ee4fcda866..16994e0909 100644
--- a/src/test/regress/expected/regproc.out
+++ b/src/test/regress/expected/regproc.out
@@ -40,6 +40,12 @@ SELECT regtype('int4');
integer
(1 row)
+SELECT regcollation('"POSIX"');
+ regcollation
+--------------
+ "POSIX"
+(1 row)
+
SELECT to_regoper('||/');
to_regoper
------------
@@ -76,6 +82,12 @@ SELECT to_regtype('int4');
integer
(1 row)
+SELECT to_regcollation('"POSIX"');
+ to_regcollation
+-----------------
+ "POSIX"
+(1 row)
+
-- with schemaname
SELECT regoper('pg_catalog.||/');
regoper
@@ -113,6 +125,12 @@ SELECT regtype('pg_catalog.int4');
integer
(1 row)
+SELECT regcollation('pg_catalog."POSIX"');
+ regcollation
+--------------
+ "POSIX"
+(1 row)
+
SELECT to_regoper('pg_catalog.||/');
to_regoper
------------
@@ -143,6 +161,12 @@ SELECT to_regtype('pg_catalog.int4');
integer
(1 row)
+SELECT to_regcollation('pg_catalog."POSIX"');
+ to_regcollation
+-----------------
+ "POSIX"
+(1 row)
+
-- schemaname not applicable
SELECT regrole('regress_regrole_test');
regrole
diff --git a/src/test/regress/sql/regproc.sql b/src/test/regress/sql/regproc.sql
index a60bc28901..5c8032800a 100644
--- a/src/test/regress/sql/regproc.sql
+++ b/src/test/regress/sql/regproc.sql
@@ -14,6 +14,7 @@ SELECT regproc('now');
SELECT regprocedure('abs(numeric)');
SELECT regclass('pg_class');
SELECT regtype('int4');
+SELECT regcollation('"POSIX"');
SELECT to_regoper('||/');
SELECT to_regoperator('+(int4,int4)');
@@ -21,6 +22,7 @@ SELECT to_regproc('now');
SELECT to_regprocedure('abs(numeric)');
SELECT to_regclass('pg_class');
SELECT to_regtype('int4');
+SELECT to_regcollation('"POSIX"');
-- with schemaname
@@ -30,12 +32,14 @@ SELECT regproc('pg_catalog.now');
SELECT regprocedure('pg_catalog.abs(numeric)');
SELECT regclass('pg_catalog.pg_class');
SELECT regtype('pg_catalog.int4');
+SELECT regcollation('pg_catalog."POSIX"');
SELECT to_regoper('pg_catalog.||/');
SELECT to_regproc('pg_catalog.now');
SELECT to_regprocedure('pg_catalog.abs(numeric)');
SELECT to_regclass('pg_catalog.pg_class');
SELECT to_regtype('pg_catalog.int4');
+SELECT to_regcollation('pg_catalog."POSIX"');
-- schemaname not applicable
--
2.20.1
v16-0004-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From 3b9ec89414e53d9b5522698a927aff2dc7d6bfee Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v16 4/7] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
25 files changed, 974 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 02779ab713..d28995dd5a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21125,7 +21125,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 660033b9c1..c0e36ef4cd 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1669,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1603,9 +1685,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1632,12 +1715,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,9 +1780,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1712,9 +1802,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1827,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1866,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1930,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2026,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2115,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2147,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2161,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2175,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2265,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2411,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2435,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2833,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..4546789446 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..369b5d43f1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3791,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index b7beb2884e..3f113499b2 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..fe14b6bd38 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 7d9fc866b9..8858ceb3df 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v16-0005-Preserve-index-dependencies-on-collation-during-.patchtext/plain; charset=us-asciiDownload
From 0671f8d9e1c27aa4311a534b831ff596b6853e60 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH v16 5/7] Preserve index dependencies on collation during
pg_upgrade.
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for collations. pg_dump will call this
function to preserve the version information for all indexes, if run with the
--binary-upgrade option.
When pg_upgrade is used to upgrade from a version of PostgreSQL that didn't
support per-object versions, the version will be recorded as unknown. If you
believe that the collation definitions haven't changed since indexes were last
built, you can use --collation-binary-compatible to tell pg_upgrade to assume
that the current versions are compatible with the existing indexes, and it will
pass --unknown-collations-binary-compatible through to pg_dump and mark them as
having the current version (according to the collation provider).
Author: Julien Rouhaud
Reviewed-by: Peter Eisentraut, Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 512 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 8e229135c2..c98ed13408 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 369b5d43f1..68898636f5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..66511b0a45 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ce31d16bd1..e3278369a2 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +388,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -704,6 +708,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6906,7 +6914,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6942,7 +6952,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6967,7 +7032,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7006,7 +7073,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7041,7 +7110,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7072,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7106,7 +7179,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7146,6 +7221,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7171,6 +7248,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16455,10 +16534,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16522,6 +16602,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16550,6 +16634,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18526,6 +18625,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0c6444ef6..e3369451a7 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -365,6 +365,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 4ef2036ecd..be2d137376 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 8858ceb3df..5bbf59ea2f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c7c25e1eae..e403107946 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10194,6 +10194,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
v16-0006-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From 4638e7a04daa7340bafde0919fd83f25c9b0e07f Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v16 6/7] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..744789b1bb 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 68898636f5..e075e9927d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,7 +3156,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8c33b67c1b..2a4378053f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -554,6 +555,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3872,6 +3874,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4039,6 +4045,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ...
+ * REFRESH VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4605,6 +4617,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17260,3 +17277,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7caf0f2f53..6c0a6a2732 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3175,6 +3175,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 804cbafda4..89fe0b38f8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2570,6 +2570,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ae35fa4aa9..43d2524cf1 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1705,7 +1720,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1751,6 +1766,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 69d163f3cc..065488374e 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 079fe1a5f3..64b3e40b70 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 09512c0f66..68f75ceaaf 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2027,6 +2027,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index c8f1a620d2..c52347fcde 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -800,6 +800,17 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v16-0007-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 15e9360f7aa78581b5fc671d2a72c1f6bdb6841a Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v16 7/7] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 20cdfabd7b..d4aa300c5a 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -936,6 +936,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</para>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Mon, Mar 16, 2020 at 03:05:20PM +0100, Julien Rouhaud wrote:
On Mon, Mar 16, 2020 at 04:57:38PM +0900, Michael Paquier wrote:
This comes from a regression test doing a sanity check to look for
catalogs which have a toastable column but no toast tables. As an
exception, it should be documented in the test's comment. Actually,
does it need to be an exception? This does not depend on
relation-level facilities so there should be no risk of recursive
dependencies, though I have not looked in details at this part.I totally missed that, and I agree that there's no need for an exception, so
fixed.
How long can actually be collation version strings? Note also
96cdeae, which makes sense for pg_depend to have one.
Regarding patch 0003, it would be nice to include some tests
independent on the rest and making use of the new functions. These
normally go in regproc.sql. For example with a collation that needs
double quotes as this is not obvious:
=# select regcollation('"POSIX"');
regcollation
--------------
"POSIX"
(1 row)On top of that, this needs tests with to_regcollation() and tests with
schema-qualified collations.Done too, using the same collation name, for both with and without schema
qualification.
It seems to me that you could add an extra test with a catalog that
does not exist, making sure that NULL is returned:
SELECT to_regtype('ng_catalog."POSIX"');
The other two cases are not really doable in regproc.sql as they would
show up the encoding used in the error message, but there could be a
point to include them in collate.icu.utf8.sql or equivalent.
Indeed. I found missing reference in datatype.sgml; func.sgml and
pgupgrade.sgml.
That looks right.
<entry>
<indexterm><primary>pg_collation_actual_version</primary></indexterm>
- <literal><function>pg_collation_actual_version(<type>oid</type>)</function></literal>
+ <literal><function>pg_collation_actual_version(<type>regcollation</type>)</function></literal>
</entry>
The function's input argument is not changed, why?Patch 0003 is visibly getting in shape, and that's an independent
piece. I guess that Thomas is looking at that, so let's wait for his
input.
Note that patch 0002 fails to compile because it is missing to include
utils/builtins.h for CStringGetTextDatum(), and that you cannot pass
down a NameData to this API, because it needs a simple char string or
you would need NameStr() or such. Anyway, it happens that you don't
need recordDependencyOnVersion() at all, because it is removed by
patch 0004 in the set, so you could just let it go.
I am still looking at the rest as of 0004~0007, the largest pieces.
--
Michael
Re: Collation versioning
On Tue, Mar 17, 2020 at 03:37:49PM +0900, Michael Paquier wrote:
On Mon, Mar 16, 2020 at 03:05:20PM +0100, Julien Rouhaud wrote:
On Mon, Mar 16, 2020 at 04:57:38PM +0900, Michael Paquier wrote:
This comes from a regression test doing a sanity check to look for
catalogs which have a toastable column but no toast tables. As an
exception, it should be documented in the test's comment. Actually,
does it need to be an exception? This does not depend on
relation-level facilities so there should be no risk of recursive
dependencies, though I have not looked in details at this part.I totally missed that, and I agree that there's no need for an exception, so
fixed.How long can actually be collation version strings? Note also
96cdeae, which makes sense for pg_depend to have one.
Versions shouldn't be that long usually, but there were some previous
discussions on how to try to come up with some workaround on systems that don't
provide a version, using a hash of the underlying file or something like that.
Using a hash value big enough to require toasting wouldn't make much sense, but
it feels safer to be ready to handle any later use, whether for collation or
other kind of objects
Regarding patch 0003, it would be nice to include some tests
independent on the rest and making use of the new functions. These
normally go in regproc.sql. For example with a collation that needs
double quotes as this is not obvious:
=# select regcollation('"POSIX"');
regcollation
--------------
"POSIX"
(1 row)On top of that, this needs tests with to_regcollation() and tests with
schema-qualified collations.Done too, using the same collation name, for both with and without schema
qualification.It seems to me that you could add an extra test with a catalog that
does not exist, making sure that NULL is returned:
SELECT to_regtype('ng_catalog."POSIX"');
Agreed, I'll add that, but using a name that looks less like a typo :)
<entry> <indexterm><primary>pg_collation_actual_version</primary></indexterm> - <literal><function>pg_collation_actual_version(<type>oid</type>)</function></literal> + <literal><function>pg_collation_actual_version(<type>regcollation</type>)</function></literal> </entry> The function's input argument is not changed, why?
That's a mistake, will fix.
Patch 0003 is visibly getting in shape, and that's an independent
piece. I guess that Thomas is looking at that, so let's wait for his
input.Note that patch 0002 fails to compile because it is missing to include
utils/builtins.h for CStringGetTextDatum(), and that you cannot pass
down a NameData to this API, because it needs a simple char string or
you would need NameStr() or such. Anyway, it happens that you don't
need recordDependencyOnVersion() at all, because it is removed by
patch 0004 in the set, so you could just let it go.
Ah good catch, I missed that during the NameData/text refactoring. I'll fix it
anyway, better to have clean history.
Re: Collation versioning
On Tue, Mar 17, 2020 at 08:19:30AM +0100, Julien Rouhaud wrote:
On Tue, Mar 17, 2020 at 03:37:49PM +0900, Michael Paquier wrote:
On Mon, Mar 16, 2020 at 03:05:20PM +0100, Julien Rouhaud wrote:
On Mon, Mar 16, 2020 at 04:57:38PM +0900, Michael Paquier wrote:
Regarding patch 0003, it would be nice to include some tests
independent on the rest and making use of the new functions. These
normally go in regproc.sql. For example with a collation that needs
double quotes as this is not obvious:
=# select regcollation('"POSIX"');
regcollation
--------------
"POSIX"
(1 row)On top of that, this needs tests with to_regcollation() and tests with
schema-qualified collations.Done too, using the same collation name, for both with and without schema
qualification.It seems to me that you could add an extra test with a catalog that
does not exist, making sure that NULL is returned:
SELECT to_regtype('ng_catalog."POSIX"');Agreed, I'll add that, but using a name that looks less like a typo :)
Tests added, including one with an error output, as the not existing schema
doesn't reveal the encoding.
Note that patch 0002 fails to compile because it is missing to include
utils/builtins.h for CStringGetTextDatum(), and that you cannot pass
down a NameData to this API, because it needs a simple char string or
you would need NameStr() or such. Anyway, it happens that you don't
need recordDependencyOnVersion() at all, because it is removed by
patch 0004 in the set, so you could just let it go.Ah good catch, I missed that during the NameData/text refactoring. I'll fix it
anyway, better to have clean history.
And this should be fixed too.
Attachments:
v16-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 2f850e4cae62c826fc99ad470ec1837cb6460206 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v16 1/7] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fc4d7f0f78..64726779d8 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,10 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ced0681ec3..2c6b1bca62 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13557,7 +13557,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13629,7 +13633,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v16-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From 5e6a7801739ace387fda9fd3eb2b1ef0f8fb91cc Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v16 2/7] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 ++++++-
src/backend/catalog/dependency.c | 11 +++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 3 ++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 57 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c2d33c76e0..4dacebf8c9 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2950,6 +2950,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</entry>
</row>
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3115,7 +3126,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index ffd52c1153..660033b9c1 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1603,7 +1603,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -1690,7 +1691,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs, NULL,
+ self_addrs->numrefs,
self_behavior);
else
{
@@ -1710,7 +1712,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs, NULL,
+ context.addrs->numrefs,
behavior);
free_object_addresses(context.addrs);
@@ -2682,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, NULL, referenced->numrefs,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 596dafe19c..312d4d3d90 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
}
/*
@@ -54,6 +55,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const char *version,
int nreferenced,
DependencyType behavior)
{
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ab5e92bdc6..473342dad2 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -184,6 +184,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const char *version,
int nreferenced,
DependencyType behavior);
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..9f2e10d428 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,9 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v16-0003-Implement-type-regcollation.patchtext/plain; charset=us-asciiDownload
From 9147fcb2947473153cfbe2193860ead5e3788cdd Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH v16 3/7] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by: Thomas Munro and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 23 +++-
doc/src/sgml/func.sgml | 36 +++---
doc/src/sgml/ref/pgupgrade.sgml | 4 +-
src/backend/utils/adt/regproc.c | 152 ++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 +++
src/include/catalog/pg_type.dat | 4 +
src/test/regress/expected/regproc.out | 40 +++++++
src/test/regress/sql/regproc.sql | 7 ++
9 files changed, 274 insertions(+), 21 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index 410eaedcb7..86d16e7b13 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4496,6 +4496,10 @@ INSERT INTO mytable VALUES(-1); -- fails
<primary>regtype</primary>
</indexterm>
+ <indexterm zone="datatype-oid">
+ <primary>regcollation</primary>
+ </indexterm>
+
<indexterm zone="datatype-oid">
<primary>regconfig</primary>
</indexterm>
@@ -4522,12 +4526,12 @@ INSERT INTO mytable VALUES(-1); -- fails
system tables.
Type <type>oid</type> represents an object identifier. There are also
- several alias types for <type>oid</type>: <type>regproc</type>,
- <type>regprocedure</type>, <type>regoper</type>, <type>regoperator</type>,
- <type>regclass</type>, <type>regtype</type>, <type>regrole</type>,
- <type>regnamespace</type>, <type>regconfig</type>, and
- <type>regdictionary</type>. <xref linkend="datatype-oid-table"/> shows an
- overview.
+ several alias types for <type>oid</type>: <type>regcollation</type>,
+ <type>regproc</type>, <type>regprocedure</type>, <type>regoper</type>,
+ <type>regoperator</type>, <type>regclass</type>, <type>regtype</type>,
+ <type>regrole</type>, <type>regnamespace</type>, <type>regconfig</type>,
+ and <type>regdictionary</type>. <xref linkend="datatype-oid-table"/> shows
+ an overview.
</para>
<para>
@@ -4591,6 +4595,13 @@ SELECT * FROM pg_attribute
<entry><literal>564182</literal></entry>
</row>
+ <row>
+ <entry><type>regcollation</type></entry>
+ <entry><structname>pg_collation</structname></entry>
+ <entry>collation name</entry>
+ <entry><literal>"POSIX"</literal></entry>
+ </row>
+
<row>
<entry><type>regproc</type></entry>
<entry><structname>pg_proc</structname></entry>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 64726779d8..fd1bba9959 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -18177,6 +18177,10 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
<primary>to_regclass</primary>
</indexterm>
+ <indexterm>
+ <primary>to_regcollation</primary>
+ </indexterm>
+
<indexterm>
<primary>to_regproc</primary>
</indexterm>
@@ -18389,6 +18393,11 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
<entry><type>regclass</type></entry>
<entry>get the OID of the named relation</entry>
</row>
+ <row>
+ <entry><literal><function>to_regcollation(<parameter>coll_name</parameter>)</function></literal></entry>
+ <entry><type>regcollation</type></entry>
+ <entry>get the OID of the named collation</entry>
+ </row>
<row>
<entry><literal><function>to_regproc(<parameter>func_name</parameter>)</function></literal></entry>
<entry><type>regproc</type></entry>
@@ -18720,20 +18729,21 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
- The <function>to_regclass</function>, <function>to_regproc</function>,
- <function>to_regprocedure</function>, <function>to_regoper</function>,
- <function>to_regoperator</function>, <function>to_regtype</function>,
- <function>to_regnamespace</function>, and <function>to_regrole</function>
- functions translate relation, function, operator, type, schema, and role
- names (given as <type>text</type>) to objects of
- type <type>regclass</type>, <type>regproc</type>, <type>regprocedure</type>,
+ The <function>to_regclass</function>, <function>to_regcollation</function>,
+ <function>to_regproc</function>, <function>to_regprocedure</function>,
+ <function>to_regoper</function>, <function>to_regoperator</function>,
+ <function>to_regtype</function>, <function>to_regnamespace</function>, and
+ <function>to_regrole</function> functions translate relation, collation,
+ function, operator, type, schema, and role names (given as
+ <type>text</type>) to objects of type <type>regclass</type>,
+ <type>regcollation</type>, <type>regproc</type>, <type>regprocedure</type>,
<type>regoper</type>, <type>regoperator</type>, <type>regtype</type>,
- <type>regnamespace</type>, and <type>regrole</type>
- respectively. These functions differ from a cast from
- text in that they don't accept a numeric OID, and that they return null
- rather than throwing an error if the name is not found (or, for
- <function>to_regproc</function> and <function>to_regoper</function>, if
- the given name matches multiple objects).
+ <type>regnamespace</type>, and <type>regrole</type> respectively. These
+ functions differ from a cast from text in that they don't accept a numeric
+ OID, and that they return null rather than throwing an error if the name is
+ not found (or, for <function>to_regproc</function> and
+ <function>to_regoper</function>, if the given name matches multiple
+ objects).
</para>
<indexterm>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6629d736b8..8e229135c2 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -773,8 +773,8 @@ psql --username=postgres --file=script.sql postgres
<para>
<application>pg_upgrade</application> does not support upgrading of databases
containing table columns using these <type>reg*</type> OID-referencing system data types:
- <type>regproc</type>, <type>regprocedure</type>, <type>regoper</type>,
- <type>regoperator</type>, <type>regconfig</type>, and
+ <type>regcollation</type>, <type>regproc</type>, <type>regprocedure</type>,
+ <type>regoper</type>, <type>regoperator</type>, <type>regconfig</type>, and
<type>regdictionary</type>. (<type>regtype</type> can be upgraded.)
</para>
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..da8cc0cf6b 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,157 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not, qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7fb574f9dc..c7c25e1eae 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6666,6 +6666,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7446,6 +7455,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b00597d6ff..ad777e37c6 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
diff --git a/src/test/regress/expected/regproc.out b/src/test/regress/expected/regproc.out
index ee4fcda866..e45ff5483f 100644
--- a/src/test/regress/expected/regproc.out
+++ b/src/test/regress/expected/regproc.out
@@ -40,6 +40,12 @@ SELECT regtype('int4');
integer
(1 row)
+SELECT regcollation('"POSIX"');
+ regcollation
+--------------
+ "POSIX"
+(1 row)
+
SELECT to_regoper('||/');
to_regoper
------------
@@ -76,6 +82,12 @@ SELECT to_regtype('int4');
integer
(1 row)
+SELECT to_regcollation('"POSIX"');
+ to_regcollation
+-----------------
+ "POSIX"
+(1 row)
+
-- with schemaname
SELECT regoper('pg_catalog.||/');
regoper
@@ -113,6 +125,12 @@ SELECT regtype('pg_catalog.int4');
integer
(1 row)
+SELECT regcollation('pg_catalog."POSIX"');
+ regcollation
+--------------
+ "POSIX"
+(1 row)
+
SELECT to_regoper('pg_catalog.||/');
to_regoper
------------
@@ -143,6 +161,12 @@ SELECT to_regtype('pg_catalog.int4');
integer
(1 row)
+SELECT to_regcollation('pg_catalog."POSIX"');
+ to_regcollation
+-----------------
+ "POSIX"
+(1 row)
+
-- schemaname not applicable
SELECT regrole('regress_regrole_test');
regrole
@@ -244,6 +268,10 @@ SELECT regtype('ng_catalog.int4');
ERROR: schema "ng_catalog" does not exist
LINE 1: SELECT regtype('ng_catalog.int4');
^
+SELECT regcollation('ng_catalog."POSIX"');
+ERROR: schema "ng_catalog" does not exist
+LINE 1: SELECT regcollation('ng_catalog."POSIX"');
+ ^
-- schemaname not applicable
SELECT regrole('regress_regrole_test');
ERROR: role "regress_regrole_test" does not exist
@@ -315,6 +343,12 @@ SELECT to_regtype('int3');
(1 row)
+SELECT to_regcollation('notacollation');
+ to_regcollation
+-----------------
+
+(1 row)
+
-- with schemaname
SELECT to_regoper('ng_catalog.||/');
to_regoper
@@ -352,6 +386,12 @@ SELECT to_regtype('ng_catalog.int4');
(1 row)
+SELECT to_regcollation('ng_catalog."POSIX"');
+ to_regcollation
+-----------------
+
+(1 row)
+
-- schemaname not applicable
SELECT to_regrole('regress_regrole_test');
to_regrole
diff --git a/src/test/regress/sql/regproc.sql b/src/test/regress/sql/regproc.sql
index a60bc28901..faab0c15ce 100644
--- a/src/test/regress/sql/regproc.sql
+++ b/src/test/regress/sql/regproc.sql
@@ -14,6 +14,7 @@ SELECT regproc('now');
SELECT regprocedure('abs(numeric)');
SELECT regclass('pg_class');
SELECT regtype('int4');
+SELECT regcollation('"POSIX"');
SELECT to_regoper('||/');
SELECT to_regoperator('+(int4,int4)');
@@ -21,6 +22,7 @@ SELECT to_regproc('now');
SELECT to_regprocedure('abs(numeric)');
SELECT to_regclass('pg_class');
SELECT to_regtype('int4');
+SELECT to_regcollation('"POSIX"');
-- with schemaname
@@ -30,12 +32,14 @@ SELECT regproc('pg_catalog.now');
SELECT regprocedure('pg_catalog.abs(numeric)');
SELECT regclass('pg_catalog.pg_class');
SELECT regtype('pg_catalog.int4');
+SELECT regcollation('pg_catalog."POSIX"');
SELECT to_regoper('pg_catalog.||/');
SELECT to_regproc('pg_catalog.now');
SELECT to_regprocedure('pg_catalog.abs(numeric)');
SELECT to_regclass('pg_catalog.pg_class');
SELECT to_regtype('pg_catalog.int4');
+SELECT to_regcollation('pg_catalog."POSIX"');
-- schemaname not applicable
@@ -70,6 +74,7 @@ SELECT regproc('ng_catalog.now');
SELECT regprocedure('ng_catalog.abs(numeric)');
SELECT regclass('ng_catalog.pg_class');
SELECT regtype('ng_catalog.int4');
+SELECT regcollation('ng_catalog."POSIX"');
-- schemaname not applicable
@@ -92,6 +97,7 @@ SELECT to_regproc('know');
SELECT to_regprocedure('absinthe(numeric)');
SELECT to_regclass('pg_classes');
SELECT to_regtype('int3');
+SELECT to_regcollation('notacollation');
-- with schemaname
@@ -101,6 +107,7 @@ SELECT to_regproc('ng_catalog.now');
SELECT to_regprocedure('ng_catalog.abs(numeric)');
SELECT to_regclass('ng_catalog.pg_class');
SELECT to_regtype('ng_catalog.int4');
+SELECT to_regcollation('ng_catalog."POSIX"');
-- schemaname not applicable
--
2.20.1
v16-0004-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From ed103d285156821f126c70579f6787d66927180c Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v16 4/7] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 130 +++++++++++-
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 19 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
25 files changed, 979 insertions(+), 69 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fd1bba9959..29e4a9b73e 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21125,7 +21125,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 660033b9c1..c0e36ef4cd 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1669,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1603,9 +1685,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1632,12 +1715,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,9 +1780,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1712,9 +1802,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1827,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1866,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1930,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2026,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2115,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2147,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2161,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2175,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2265,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2411,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2435,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2833,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..4546789446 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..369b5d43f1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3791,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 312d4d3d90..3f113499b2 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,12 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const char *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -545,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..fe14b6bd38 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 473342dad2..8858ceb3df 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,17 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const char *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v16-0005-Preserve-index-dependencies-on-collation-during-.patchtext/plain; charset=us-asciiDownload
From c2af86f93f0c20ad49411fb8b4b4fd553e46f44a Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH v16 5/7] Preserve index dependencies on collation during
pg_upgrade.
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for collations. pg_dump will call this
function to preserve the version information for all indexes, if run with the
--binary-upgrade option.
When pg_upgrade is used to upgrade from a version of PostgreSQL that didn't
support per-object versions, the version will be recorded as unknown. If you
believe that the collation definitions haven't changed since indexes were last
built, you can use --collation-binary-compatible to tell pg_upgrade to assume
that the current versions are compatible with the existing indexes, and it will
pass --unknown-collations-binary-compatible through to pg_dump and mark them as
having the current version (according to the collation provider).
Author: Julien Rouhaud
Reviewed-by: Peter Eisentraut, Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 192 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 512 insertions(+), 72 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 8e229135c2..c98ed13408 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 369b5d43f1..68898636f5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..66511b0a45 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2c6b1bca62..9332f15623 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +388,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -704,6 +708,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6907,7 +6915,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6943,7 +6953,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6968,7 +7033,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7007,7 +7074,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7042,7 +7111,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7073,7 +7144,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7107,7 +7180,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7147,6 +7222,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7172,6 +7249,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16456,10 +16535,11 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
- * will have ensured the constraint is emitted first.) Note that the
- * emitted comment has to be shown as depending on the constraint, not the
- * index, in such cases.
+ * do dump any comment, or in binary upgrade mode dependency on a collation
+ * version for it. (This is safe because dependency ordering will have
+ * ensured the constraint is emitted first.) Note that the emitted
+ * comment has to be shown as depending on the constraint, not the index,
+ * in such cases.
*/
if (!is_constraint)
{
@@ -16523,6 +16603,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16551,6 +16635,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18527,6 +18626,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0c6444ef6..e3369451a7 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -365,6 +365,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 4ef2036ecd..be2d137376 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 8858ceb3df..5bbf59ea2f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index c619d02465..69d163f3cc 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c7c25e1eae..e403107946 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10194,6 +10194,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
v16-0006-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From 0947bb1e0c02d971fded7e543f77d4577ca48b53 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v16 6/7] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..744789b1bb 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 68898636f5..e075e9927d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,7 +3156,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8c33b67c1b..2a4378053f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -554,6 +555,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3872,6 +3874,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4039,6 +4045,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ...
+ * REFRESH VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4605,6 +4617,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17260,3 +17277,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7caf0f2f53..6c0a6a2732 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3175,6 +3175,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 804cbafda4..89fe0b38f8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2570,6 +2570,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ae35fa4aa9..43d2524cf1 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1705,7 +1720,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1751,6 +1766,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 69d163f3cc..065488374e 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 079fe1a5f3..64b3e40b70 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 09512c0f66..68f75ceaaf 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2027,6 +2027,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index c8f1a620d2..c52347fcde 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -800,6 +800,17 @@ REINDEX TABLE collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v16-0007-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 23021f31feff00f7c4b8aa81b42d7b69f4cbd3ed Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v16 7/7] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 20cdfabd7b..d4aa300c5a 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -936,6 +936,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</para>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
<div dir="ltr">njhjo<br>
</div><div dir="ltr"><br>
</div><div dir="ltr"><br>
</div><div dir="ltr"><br>
</div><div class="wps_signature">Enviado desde mi Redmi 4A</div><div class="wps_quotion">El Julien Rouhaud <rjuju123@gmail.com>, 16 mar. 2020 3:05 p. m. escribió:<br type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p></p><p dir="ltr">On Mon, Mar 16, 2020 at 04:57:38PM +0900, Michael Paquier wrote:
<br>
> On Thu, Mar 12, 2020 at 03:00:26PM +0100, Julien Rouhaud wrote:
<br>
> > And v15 due to conflict with b08dee24a5 (Add pg_dump support for ALTER obj
<br>
> > DEPENDS ON EXTENSION).
<br>
>
<br>
> I have looked at patches 0001~0003 in the set for now.
<br>
<br>
<br>
Thanks!
<br>
<br>
<br>
> In patch 0002, you have the following addition:
<br>
> @@ -103,9 +103,10 @@ ORDER BY 1, 2;
<br>
> pg_class | relacl | aclitem[]
<br>
> pg_class | reloptions | text[]
<br>
> pg_class | relpartbound | pg_node_tree
<br>
> + pg_depend | refobjversion | text
<br>
> This comes from a regression test doing a sanity check to look for
<br>
> catalogs which have a toastable column but no toast tables. As an
<br>
> exception, it should be documented in the test's comment. Actually,
<br>
> does it need to be an exception? This does not depend on
<br>
> relation-level facilities so there should be no risk of recursive
<br>
> dependencies, though I have not looked in details at this part.
<br>
<br>
<br>
I totally missed that, and I agree that there's no need for an exception, so
<br>
fixed.
<br>
<br>
<br>
> + <para>
<br>
> + The only current use of <structfield>refobjversion</structfield> is to
<br>
> + record dependencies between indexes and collation versions.
<br>
> + </para>
<br>
> [...]
<br>
> + <row>
<br>
> + <entry><structfield>refobjversion</structfield></entry>
<br>
> + <entry><type>text</type></entry>
<br>
> + <entry></entry>
<br>
> + <entry>
<br>
> + An optional version for the referenced object; see text
<br>
> + </entry>
<br>
> + </row>
<br>
> Couldn't you merge both paragraphs here?
<br>
<br>
<br>
Done.
<br>
<br>
<br>
> Regarding patch 0003, it would be nice to include some tests
<br>
> independent on the rest and making use of the new functions. These
<br>
> normally go in regproc.sql. For example with a collation that needs
<br>
> double quotes as this is not obvious:
<br>
> =# select regcollation('"POSIX"');
<br>
> regcollation
<br>
> --------------
<br>
> "POSIX"
<br>
> (1 row)
<br>
>
<br>
> On top of that, this needs tests with to_regcollation() and tests with
<br>
> schema-qualified collations.
<br>
<br>
<br>
Done too, using the same collation name, for both with and without schema
<br>
qualification.
<br>
<br>
<br>
> Documentation for to_regcollation() is missing. And it looks that
<br>
> many parts of the documentation are missing an update. One example in
<br>
> datatype.sgml:
<br>
> Type <type>oid</type> represents an object identifier. There are also
<br>
> several alias types for <type>oid</type>: <type>regproc</type>,
<br>
> <type>regprocedure</type>, <type>regoper</type>, <type>regoperator</type>,
<br>
> <type>regclass</type>, <type>regtype</type>, <type>regrole</type>,
<br>
> <type>regnamespace</type>, <type>regconfig</type>, and
<br>
> <type>regdictionary</type>. <xref linkend="datatype-oid-table"/> shows an
<br>
> overview.
<br>
> At quick glance, there are more sections in need of a refresh..
<br>
<br>
<br>
Indeed. I found missing reference in datatype.sgml; func.sgml and
<br>
pgupgrade.sgml.
<br>
<br>
v16 attached.
<br>
</p>
</blockquote></div>
Re: Collation versioning
Did we discuss the regcollation type? In the current patch set, it's
only used in two places in a new regression test, where it can easily be
replaced by a join. Do we need it?
I realize we've been adding new reg* types lately; I'm not sure what the
current idea is on that.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
Re: Peter Eisentraut 2020-03-17 <fd8d4475-85ad-506f-2dda-f4d6e66785bc@2ndquadrant.com>
Did we discuss the regcollation type? In the current patch set, it's only
used in two places in a new regression test, where it can easily be replaced
by a join. Do we need it?I realize we've been adding new reg* types lately; I'm not sure what the
current idea is on that.
Not sure if that's the case there, but reg* typecasts are very handy
when used interactively in ad-hoc queries.
Christoph
Re: Collation versioning
On Tue, Mar 17, 2020 at 05:31:47PM +0100, Christoph Berg wrote:
Re: Peter Eisentraut 2020-03-17 <fd8d4475-85ad-506f-2dda-f4d6e66785bc@2ndquadrant.com>
Did we discuss the regcollation type? In the current patch set, it's only
used in two places in a new regression test, where it can easily be replaced
by a join. Do we need it?
I originally wrote it for a previous version of the patchset, to shorten the
pg_dump query, but that went out when I replaced the DDL command with native
functions instead. It didn't seem to hurt to keep it, so I relied on it in the
regression tests.
I realize we've been adding new reg* types lately; I'm not sure what the
current idea is on that.Not sure if that's the case there, but reg* typecasts are very handy
when used interactively in ad-hoc queries.
+1. I'm obviously biased, but I find it quite useful when querying pg_depend,
which may become more frequent once we start generating warnings about possibly
corrupted indexes.
Re: Collation versioning
On Tue, Mar 17, 2020 at 06:43:51PM +0100, Julien Rouhaud wrote:
On Tue, Mar 17, 2020 at 05:31:47PM +0100, Christoph Berg wrote:
Not sure if that's the case there, but reg* typecasts are very handy
when used interactively in ad-hoc queries.+1. I'm obviously biased, but I find it quite useful when querying pg_depend,
which may become more frequent once we start generating warnings about possibly
corrupted indexes.
That means less joins for lookup queries, and collations can be
schema-qualified, so I would be in favor of adding it rather than not.
Now, it is true as well that ::regcollation is not a mandatory
requirement for the feature discussed on this thread.
--
Michael
Re: Collation versioning
On Tue, Mar 17, 2020 at 11:42:34AM +0100, Julien Rouhaud wrote:
On Tue, Mar 17, 2020 at 08:19:30AM +0100, Julien Rouhaud wrote:
On Tue, Mar 17, 2020 at 03:37:49PM +0900, Michael Paquier wrote:
It seems to me that you could add an extra test with a catalog that
does not exist, making sure that NULL is returned:
SELECT to_regtype('ng_catalog."POSIX"');Agreed, I'll add that, but using a name that looks less like a typo :)
Tests added, including one with an error output, as the not existing schema
doesn't reveal the encoding.
Yep.
Ah good catch, I missed that during the NameData/text refactoring. I'll fix it
anyway, better to have clean history.And this should be fixed too.
Thanks.
It would be good to be careful about the indentation. Certain parts
of 0003 don't respect the core indentation. Not necessarily your job
though. Other than that 0003 seems to be in good shape.
@@ -54,6 +55,7 @@ recordDependencyOn(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
+ const char *version,
int nreferenced,
DependencyType behavior)
Nit from patch 0002: the argument "version" should be fourth I think,
keeping the number of referenced objects and the referenced objects
close. And actually, this "version" argument is removed in patch
0004, replaced by the boolean track_version. (By reading the
arguments below I'd rather keep *version).So, 0004 is the core of the changes. I have found a bug with the
handling of refobjversion and pg_depend entries. When swapping the
dependencies of the old and new indexes in index_concurrently_swap(),
refobjversion remains set to the value of the old index. I used a
manual UPDATE on pg_depend to emulate that with a past fake version
string to emulate that (sneaky I know), but you would get the same
behavior with an upgraded instance. refobjversion should be updated
to the version of the new index.
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
Incorrect declaration format.+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
I don't think that the API is right for this stuff, as you introduce
collation-level knowledge into something which has been much more
flexible until now. Wouldn't it be better to move the refobjversion
string directly into ObjectAddress?+ * Test if a record exists for the given depender and referenceds addresses.
[...]
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
Typos.+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
What are disappearing version strings?+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
Should this explain why?+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
Comment block format.+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
Why is that? The code in 0004 has no mention of that, and relies on
this code path:
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
And how is that even extensible for custom index AMs? There are other
things than bloom out there.+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.=
+ */
[...]
+ /*
+ * do not issue UNKNOWN VERSION is caller specified that those are
+ * compatible
+ */
Typos from patch 5.- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
This needs to actually be shaped with two separate arguments for
--cluster-name or using quotes would not work properly if I recall
correctly. Not your patch's fault, so I would fix that separately.
--
MichaelRe: Collation versioning
On Wed, Mar 18, 2020 at 04:55:25PM +0900, Michael Paquier wrote:
On Tue, Mar 17, 2020 at 11:42:34AM +0100, Julien Rouhaud wrote:
On Tue, Mar 17, 2020 at 08:19:30AM +0100, Julien Rouhaud wrote:
It would be good to be careful about the indentation. Certain parts
of 0003 don't respect the core indentation. Not necessarily your job
though. Other than that 0003 seems to be in good shape.
I'll try to do a partial pgindent run on all patches before next patchset.
@@ -54,6 +55,7 @@ recordDependencyOn(const ObjectAddress *depender, void recordMultipleDependencies(const ObjectAddress *depender, const ObjectAddress *referenced, + const char *version, int nreferenced, DependencyType behavior) Nit from patch 0002: the argument "version" should be fourth I think, keeping the number of referenced objects and the referenced objects close. And actually, this "version" argument is removed in patch 0004, replaced by the boolean track_version. (By reading the arguments below I'd rather keep *version).So, 0004 is the core of the changes. I have found a bug with the
handling of refobjversion and pg_depend entries. When swapping the
dependencies of the old and new indexes in index_concurrently_swap(),
refobjversion remains set to the value of the old index. I used a
manual UPDATE on pg_depend to emulate that with a past fake version
string to emulate that (sneaky I know), but you would get the same
behavior with an upgraded instance. refobjversion should be updated
to the version of the new index.
Oh good catch. I'll dig into it.
+ if (track_version) + { + /* Only dependency on a collation is supported. */ + if (referenced->classId == CollationRelationId) + { + /* C and POSIX collations don't require tracking the version */ + if (referenced->objectId == C_COLLATION_OID || + referenced->objectId == POSIX_COLLATION_OID) + continue; I don't think that the API is right for this stuff, as you introduce collation-level knowledge into something which has been much more flexible until now. Wouldn't it be better to move the refobjversion string directly into ObjectAddress?
We could, but we would then need to add code to retrieve the collation version
in multiple places (at least RecordDependencyOnCollation and
recordDependencyOnSingleRelExpr). I'm afraid that'll open room for bugs if
some other places are missed, now or later, even more if more objects get a
versionning support.
+ * Test if a record exists for the given depender and referenceds addresses. [...] + /* recordDependencyOnSingleRelExpr get rid of duplicate entries */ Typos.+ /* XXX should we warn about "disappearing" versions? */ + if (current_version) What are disappearing version strings?
A collation for which a version was previously recorded but that now doesn't
report a version anymore. For instance if upgrading from glibc X.Y to X.Z
changes gnu_get_libc_version() to return NULL, or if a new major pg version
removes support for glibc (or other lib) versioning. It seems unlikely to
happen, and if that happens there's nothing we can do anymore to warn about
possible corruption anyway.
+ /* + * Perform version sanity checks on the relation underlying indexes if + * it's not a VACUUM FULL + */ + if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) && + onerel->rd_rel->relhasindex) Should this explain why?
I was assuming it's self explanatory, since VACUUM FULL is one of the 3 only
ways to fix a possibly corrupt index (on top of REINDEX and ALTER INDEX ...
ALTER COLLATION ... REFRESH VERSION). I can mention it if needed though.
+ /* Record collations from the type itself, or underlying in case of + * complex type. Note that if the direct parent is a CollateExpr + * node, there's no need to record the type underlying collation if Comment block format.
Oops, will fix.
+-- for key columns, hash indexes should record dependency on the collation but +-- not the version +CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es); Why is that?
Because hash indexes don't rely on the sort order for the key columns? So even
if the sort order changes the index won't get corrupted (unless it's a non
deterministic collation of course).
The code in 0004 has no mention of that, and relies on this code path: +/* + * Returns whether the given index access method depend on a stable collation + * order. + */ +static bool +index_depends_stable_coll_order(Oid amoid) +{ + return (amoid != HASH_AM_OID && + strcmp(get_am_name(amoid), "bloom") != 0); +}
This is handled in index_create():
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);And how is that even extensible for custom index AMs? There are other
things than bloom out there.
That's not extensible, and that was discussed a month ago at
/messages/by-id/CA+hUKGJ-TqYomCAYgJt53_0b9KmfSyD2qW59xfzmZa3ftRJFzA@mail.gmail.com.
Thomas was in favor of handling that at the operator level, but it seems to me
that this would require quite a lot of extra work (assuming that we'd need to
find if the opclass handles any operator different from BTEqualStrategyNumber),
but I'm still not sure that it's sufficient to prove that an AM doesn't
internally rely on a stable ordering or not. So I'd be in favor of adding a
new field in IndexAmRoutine for that. Without consensus on that, I choose the
quickest approach. Do you have any opinion on that?
- $self->logfile, '-o', "--cluster-name=$name", 'start'); + $self->logfile, '-o', $options, 'start'); This needs to actually be shaped with two separate arguments for --cluster-name or using quotes would not work properly if I recall correctly. Not your patch's fault, so I would fix that separately.
Ok!
Re: Collation versioning
On Wed, Mar 18, 2020 at 09:56:40AM +0100, Julien Rouhaud wrote:
On Wed, Mar 18, 2020 at 04:55:25PM +0900, Michael Paquier wrote:
It would be good to be careful about the indentation. Certain parts
of 0003 don't respect the core indentation. Not necessarily your job
though. Other than that 0003 seems to be in good shape.I'll try to do a partial pgindent run on all patches before next patchset.
I run a pgindent on all .[ch] files and kept only the relevant changes, for
each patch, so this should now be fine.
@@ -54,6 +55,7 @@ recordDependencyOn(const ObjectAddress *depender, void recordMultipleDependencies(const ObjectAddress *depender, const ObjectAddress *referenced, + const char *version, int nreferenced, DependencyType behavior) Nit from patch 0002: the argument "version" should be fourth I think, keeping the number of referenced objects and the referenced objects close. And actually, this "version" argument is removed in patch 0004, replaced by the boolean track_version. (By reading the arguments below I'd rather keep *version).
I changed 0002 to have the version as the 4th argument just in case.
So, 0004 is the core of the changes. I have found a bug with the
handling of refobjversion and pg_depend entries. When swapping the
dependencies of the old and new indexes in index_concurrently_swap(),
refobjversion remains set to the value of the old index. I used a
manual UPDATE on pg_depend to emulate that with a past fake version
string to emulate that (sneaky I know), but you would get the same
behavior with an upgraded instance. refobjversion should be updated
to the version of the new index.Oh good catch. I'll dig into it.
AFAICT it was only missing a call to index_update_collation_versions() in
ReindexRelationConcurrently. I added regression tests to make sure that
REINDEX, REINDEX [INDEX|TABLE] CONCURRENTLY and VACUUM FULL are doing what's
expected.
Given discussion in nearby threads, I obviously can't add tests for failed
REINDEX CONCURRENTLY, so here's what's happening with a manual repro:
=# CREATE TABLE t1(id integer, val text);
CREATE
=# CREATE INDEX ON t1(val COLLATE "fr-x-icu");
CREATE
=# UPDATE pg_depend SET refobjversion = 'meh' WHERE refobjversion = '153.97';
UPDATE 1
=# REINDEX TABLE CONCURRENTLY t1 ;
LOCATION: ReindexRelationConcurrently, indexcmds.c:2839
^CCancel request sent
ERROR: 57014: canceling statement due to user request
LOCATION: ProcessInterrupts, postgres.c:3171
=# SELECT objid::regclass, indisvalid, refobjversion
FROM pg_depend d
JOIN pg_index i ON i.indexrelid = d.objid
WHERE refobjversion IS NOT NULL;
objid | indisvalid | refobjversion
------------------+------------+---------------
t1_val_idx_ccold | f | 153.97
t1_val_idx | t | meh
(2 rows)
=# REINDEX TABLE t1;
WARNING: 0A000: cannot reindex invalid index "pg_toast.pg_toast_16418_index_ccold" on TOAST table, skipping
LOCATION: reindex_relation, index.c:3987
REINDEX
=# SELECT objid::regclass, indisvalid, refobjversion
FROM pg_depend d
JOIN pg_index i ON i.indexrelid = d.objid
WHERE refobjversion IS NOT NULL;
objid | indisvalid | refobjversion
------------------+------------+---------------
t1_val_idx_ccold | t | 153.97
t1_val_idx | t | 153.97
(2 rows)
ISTM that it's working as intended.
+ if (track_version) + { + /* Only dependency on a collation is supported. */ + if (referenced->classId == CollationRelationId) + { + /* C and POSIX collations don't require tracking the version */ + if (referenced->objectId == C_COLLATION_OID || + referenced->objectId == POSIX_COLLATION_OID) + continue; I don't think that the API is right for this stuff, as you introduce collation-level knowledge into something which has been much more flexible until now. Wouldn't it be better to move the refobjversion string directly into ObjectAddress?We could, but we would then need to add code to retrieve the collation version
in multiple places (at least RecordDependencyOnCollation and
recordDependencyOnSingleRelExpr). I'm afraid that'll open room for bugs if
some other places are missed, now or later, even more if more objects get a
versionning support.
No change here.
+ /* + * Perform version sanity checks on the relation underlying indexes if + * it's not a VACUUM FULL + */ + if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) && + onerel->rd_rel->relhasindex) Should this explain why?
Explanation added.
v17 attached, rebased against master (conflict since 8408e3a557).
Attachments:
v17-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 1b35848b0b3589f6216ae22c3621859e37d65899 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v17 1/7] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c6f95fa688..c2d33c76e0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 2a8683a734..710b51ff7c 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21113,10 +21113,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ced0681ec3..2c6b1bca62 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13557,7 +13557,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13629,7 +13633,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v17-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From ce3c3c46ddb9ac1d5f3eeafa8d749281ce98806a Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v17 2/7] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 ++++++-
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 61 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c2d33c76e0..4dacebf8c9 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2950,6 +2950,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</entry>
</row>
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3115,7 +3126,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index ffd52c1153..281f5b7c28 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1603,7 +1603,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1690,7 +1692,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1710,7 +1714,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2682,7 +2688,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 596dafe19c..84a0b99311 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ab5e92bdc6..62e86a19b6 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -185,6 +185,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v17-0003-Implement-type-regcollation.patchtext/plain; charset=us-asciiDownload
From 0d6c5f4757ef35a61eccf94987c0afee9a16d73c Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 5 Dec 2019 18:59:28 +0100
Subject: [PATCH v17 3/7] Implement type regcollation.
This will be helpful for a following commit.
Author: Julien Rouhaud
Reviewed-by: Thomas Munro and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 14 +++
doc/src/sgml/func.sgml | 32 ++++--
doc/src/sgml/ref/pgupgrade.sgml | 1 +
src/backend/utils/adt/regproc.c | 153 ++++++++++++++++++++++++++
src/include/catalog/pg_cast.dat | 14 +++
src/include/catalog/pg_proc.dat | 15 +++
src/include/catalog/pg_type.dat | 4 +
src/test/regress/expected/regproc.out | 40 +++++++
src/test/regress/sql/regproc.sql | 7 ++
9 files changed, 269 insertions(+), 11 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index 157fe4e727..ffe497a63a 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4476,6 +4476,10 @@ INSERT INTO mytable VALUES(-1); -- fails
<primary>regclass</primary>
</indexterm>
+ <indexterm zone="datatype-oid">
+ <primary>regcollation</primary>
+ </indexterm>
+
<indexterm zone="datatype-oid">
<primary>regconfig</primary>
</indexterm>
@@ -4496,6 +4500,10 @@ INSERT INTO mytable VALUES(-1); -- fails
<primary>regoperator</primary>
</indexterm>
+ <indexterm zone="datatype-oid">
+ <primary>regtype</primary>
+ </indexterm>
+
<indexterm zone="datatype-oid">
<primary>regproc</primary>
</indexterm>
@@ -4602,6 +4610,12 @@ SELECT * FROM pg_attribute
<entry><literal>pg_type</literal></entry>
</row>
+ <row>
+ <entry><type>regcollation</type></entry>
+ <entry><structname>pg_collation</structname></entry>
+ <entry>collation name</entry>
+ <entry><literal>"POSIX"</literal></entry>
+ </row>
<row>
<entry><type>regconfig</type></entry>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 710b51ff7c..b84502295e 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -18177,6 +18177,10 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
<primary>to_regclass</primary>
</indexterm>
+ <indexterm>
+ <primary>to_regcollation</primary>
+ </indexterm>
+
<indexterm>
<primary>to_regnamespace</primary>
</indexterm>
@@ -18389,6 +18393,11 @@ SELECT pg_type_is_visible('myschema.widget'::regtype);
<entry><type>regclass</type></entry>
<entry>get the OID of the named relation</entry>
</row>
+ <row>
+ <entry><literal><function>to_regcollation(<parameter>coll_name</parameter>)</function></literal></entry>
+ <entry><type>regcollation</type></entry>
+ <entry>get the OID of the named collation</entry>
+ </row>
<row>
<entry><literal><function>to_regnamespace(<parameter>schema_name</parameter>)</function></literal></entry>
<entry><type>regnamespace</type></entry>
@@ -18721,17 +18730,18 @@ SELECT collation for ('foo' COLLATE "de_DE");
<para>
The functions <function>to_regclass</function>,
- <function>to_regnamespace</function>, <function>to_regoper</function>,
- <function>to_regoperator</function>, <function>to_regrole</function>,
- <function>to_regproc</function>, <function>to_regprocedure</function>, and
- <function>to_regtype</function>, functions translate relation, schema,
- operator, role, function, and type names (given as <type>text</type>) to
- objects of the corresponding <type>reg*</type> type (see <xref
- linkend="datatype-oid"/> about the types). These functions differ from a
- cast from text in that they don't accept a numeric OID, and that they
- return null rather than throwing an error if the name is not found (or, for
- <function>to_regproc</function> and <function>to_regoper</function>, if the
- given name matches multiple objects).
+ <function>to_regcollation</function>, <function>to_regnamespace</function>,
+ <function>to_regoper</function>, <function>to_regoperator</function>,
+ <function>to_regrole</function>, <function>to_regproc</function>,
+ <function>to_regprocedure</function>, and <function>to_regtype</function>,
+ functions translate relation, collation, schema, operator, role, function,
+ and type names (given as <type>text</type>) to objects of the corresponding
+ <type>reg*</type> type (see <xref linkend="datatype-oid"/> about the types).
+ These functions differ from a cast from text in that they don't accept a
+ numeric OID, and that they return null rather than throwing an error if the
+ name is not found (or, for <function>to_regproc</function> and
+ <function>to_regoper</function>, if the given name matches multiple
+ objects).
</para>
<indexterm>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 9e4b2d69a4..5e396f41fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -775,6 +775,7 @@ psql --username=postgres --file=script.sql postgres
containing table columns using these <type>reg*</type> OID-referencing system data types:
<simplelist>
<member><type>regconfig</type></member>
+ <member><type>regcollation</type></member>
<member><type>regdictionary</type></member>
<member><type>regnamespace</type></member>
<member><type>regoper</type></member>
diff --git a/src/backend/utils/adt/regproc.c b/src/backend/utils/adt/regproc.c
index f0fa52bc27..c800d797ac 100644
--- a/src/backend/utils/adt/regproc.c
+++ b/src/backend/utils/adt/regproc.c
@@ -24,6 +24,7 @@
#include "access/htup_details.h"
#include "catalog/namespace.h"
#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_operator.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_ts_config.h"
@@ -1043,6 +1044,158 @@ regclasssend(PG_FUNCTION_ARGS)
}
+/*
+ * regcollationin - converts "collationname" to collation OID
+ *
+ * We also accept a numeric OID, for symmetry with the output routine.
+ *
+ * '-' signifies unknown (OID 0). In all other cases, the input must
+ * match an existing pg_collation entry.
+ */
+Datum
+regcollationin(PG_FUNCTION_ARGS)
+{
+ char *collation_name_or_oid = PG_GETARG_CSTRING(0);
+ Oid result = InvalidOid;
+ List *names;
+
+ /* '-' ? */
+ if (strcmp(collation_name_or_oid, "-") == 0)
+ PG_RETURN_OID(InvalidOid);
+
+ /* Numeric OID? */
+ if (collation_name_or_oid[0] >= '0' &&
+ collation_name_or_oid[0] <= '9' &&
+ strspn(collation_name_or_oid, "0123456789") == strlen(collation_name_or_oid))
+ {
+ result = DatumGetObjectId(DirectFunctionCall1(oidin,
+ CStringGetDatum(collation_name_or_oid)));
+ PG_RETURN_OID(result);
+ }
+
+ /* Else it's a name, possibly schema-qualified */
+
+ /* The rest of this wouldn't work in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ elog(ERROR, "regcollation values must be OIDs in bootstrap mode");
+
+ /*
+ * Normal case: parse the name into components and see if it matches any
+ * pg_collation entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name_or_oid);
+
+ result = get_collation_oid(names, false);
+
+ PG_RETURN_OID(result);
+}
+
+/*
+ * to_regcollation - converts "collationname" to collation OID
+ *
+ * If the name is not found, we return NULL.
+ */
+Datum
+to_regcollation(PG_FUNCTION_ARGS)
+{
+ char *collation_name = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ Oid result;
+ List *names;
+
+ /*
+ * Parse the name into components and see if it matches any pg_collation
+ * entries in the current search path.
+ */
+ names = stringToQualifiedNameList(collation_name);
+
+ /* We might not even have permissions on this relation; don't lock it. */
+ result = get_collation_oid(names, true);
+
+ if (OidIsValid(result))
+ PG_RETURN_OID(result);
+ else
+ PG_RETURN_NULL();
+}
+
+/*
+ * regcollationout - converts collation OID to "collation_name"
+ */
+Datum
+regcollationout(PG_FUNCTION_ARGS)
+{
+ Oid collationid = PG_GETARG_OID(0);
+ char *result;
+ HeapTuple collationtup;
+
+ if (collationid == InvalidOid)
+ {
+ result = pstrdup("-");
+ PG_RETURN_CSTRING(result);
+ }
+
+ collationtup = SearchSysCache1(COLLOID, ObjectIdGetDatum(collationid));
+
+ if (HeapTupleIsValid(collationtup))
+ {
+ Form_pg_collation collationform = (Form_pg_collation) GETSTRUCT(collationtup);
+ char *collationname = NameStr(collationform->collname);
+
+ /*
+ * In bootstrap mode, skip the fancy namespace stuff and just return
+ * the collation name. (This path is only needed for debugging output
+ * anyway.)
+ */
+ if (IsBootstrapProcessingMode())
+ result = pstrdup(collationname);
+ else
+ {
+ char *nspname;
+
+ /*
+ * Would this collation be found by regcollationin? If not,
+ * qualify it.
+ */
+ if (CollationIsVisible(collationid))
+ nspname = NULL;
+ else
+ nspname = get_namespace_name(collationform->collnamespace);
+
+ result = quote_qualified_identifier(nspname, collationname);
+ }
+
+ ReleaseSysCache(collationtup);
+ }
+ else
+ {
+ /* If OID doesn't match any pg_collation entry, return it numerically */
+ result = (char *) palloc(NAMEDATALEN);
+ snprintf(result, NAMEDATALEN, "%u", collationid);
+ }
+
+ PG_RETURN_CSTRING(result);
+}
+
+/*
+ * regcollationrecv - converts external binary format to regcollation
+ */
+Datum
+regcollationrecv(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidrecv, so share code */
+ return oidrecv(fcinfo);
+}
+
+/*
+ * regcollationsend - converts regcollation to binary format
+ */
+Datum
+regcollationsend(PG_FUNCTION_ARGS)
+{
+ /* Exactly the same as oidsend, so share code */
+ return oidsend(fcinfo);
+}
+
+
/*
* regtypein - converts "typename" to type OID
*
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 6ef8b8a4e7..01c5328ddd 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -189,6 +189,20 @@
castcontext => 'a', castmethod => 'f' },
{ castsource => 'regclass', casttarget => 'int4', castfunc => '0',
castcontext => 'a', castmethod => 'b' },
+{ castsource => 'oid', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'oid', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'int8', casttarget => 'regcollation', castfunc => 'oid',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int2', casttarget => 'regcollation', castfunc => 'int4(int2)',
+ castcontext => 'i', castmethod => 'f' },
+{ castsource => 'int4', casttarget => 'regcollation', castfunc => '0',
+ castcontext => 'i', castmethod => 'b' },
+{ castsource => 'regcollation', casttarget => 'int8', castfunc => 'int8(oid)',
+ castcontext => 'a', castmethod => 'f' },
+{ castsource => 'regcollation', casttarget => 'int4', castfunc => '0',
+ castcontext => 'a', castmethod => 'b' },
{ castsource => 'oid', casttarget => 'regtype', castfunc => '0',
castcontext => 'i', castmethod => 'b' },
{ castsource => 'regtype', casttarget => 'oid', castfunc => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7fb574f9dc..c7c25e1eae 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6666,6 +6666,15 @@
{ oid => '3495', descr => 'convert classname to regclass',
proname => 'to_regclass', provolatile => 's', prorettype => 'regclass',
proargtypes => 'text', prosrc => 'to_regclass' },
+{ oid => '9508', descr => 'I/O',
+ proname => 'regcollationin', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'cstring', prosrc => 'regcollationin' },
+{ oid => '9509', descr => 'I/O',
+ proname => 'regcollationout', provolatile => 's', prorettype => 'cstring',
+ proargtypes => 'regcollation', prosrc => 'regcollationout' },
+{ oid => '9510', descr => 'convert classname to regcollation',
+ proname => 'to_regcollation', provolatile => 's', prorettype => 'regcollation',
+ proargtypes => 'text', prosrc => 'to_regcollation' },
{ oid => '2220', descr => 'I/O',
proname => 'regtypein', provolatile => 's', prorettype => 'regtype',
proargtypes => 'cstring', prosrc => 'regtypein' },
@@ -7446,6 +7455,12 @@
{ oid => '2453', descr => 'I/O',
proname => 'regclasssend', prorettype => 'bytea', proargtypes => 'regclass',
prosrc => 'regclasssend' },
+{ oid => '9511', descr => 'I/O',
+ proname => 'regcollationrecv', prorettype => 'regcollation',
+ proargtypes => 'internal', prosrc => 'regcollationrecv' },
+{ oid => '9512', descr => 'I/O',
+ proname => 'regcollationsend', prorettype => 'bytea', proargtypes => 'regcollation',
+ prosrc => 'regcollationsend' },
{ oid => '2454', descr => 'I/O',
proname => 'regtyperecv', prorettype => 'regtype', proargtypes => 'internal',
prosrc => 'regtyperecv' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b00597d6ff..ad777e37c6 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -379,6 +379,10 @@
typname => 'regclass', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regclassin', typoutput => 'regclassout',
typreceive => 'regclassrecv', typsend => 'regclasssend', typalign => 'i' },
+{ oid => '9506', array_type_oid => '9507', descr => 'registered collation',
+ typname => 'regcollation', typlen => '4', typbyval => 't', typcategory => 'N',
+ typinput => 'regcollationin', typoutput => 'regcollationout',
+ typreceive => 'regcollationrecv', typsend => 'regcollationsend', typalign => 'i' },
{ oid => '2206', array_type_oid => '2211', descr => 'registered type',
typname => 'regtype', typlen => '4', typbyval => 't', typcategory => 'N',
typinput => 'regtypein', typoutput => 'regtypeout',
diff --git a/src/test/regress/expected/regproc.out b/src/test/regress/expected/regproc.out
index ee4fcda866..e45ff5483f 100644
--- a/src/test/regress/expected/regproc.out
+++ b/src/test/regress/expected/regproc.out
@@ -40,6 +40,12 @@ SELECT regtype('int4');
integer
(1 row)
+SELECT regcollation('"POSIX"');
+ regcollation
+--------------
+ "POSIX"
+(1 row)
+
SELECT to_regoper('||/');
to_regoper
------------
@@ -76,6 +82,12 @@ SELECT to_regtype('int4');
integer
(1 row)
+SELECT to_regcollation('"POSIX"');
+ to_regcollation
+-----------------
+ "POSIX"
+(1 row)
+
-- with schemaname
SELECT regoper('pg_catalog.||/');
regoper
@@ -113,6 +125,12 @@ SELECT regtype('pg_catalog.int4');
integer
(1 row)
+SELECT regcollation('pg_catalog."POSIX"');
+ regcollation
+--------------
+ "POSIX"
+(1 row)
+
SELECT to_regoper('pg_catalog.||/');
to_regoper
------------
@@ -143,6 +161,12 @@ SELECT to_regtype('pg_catalog.int4');
integer
(1 row)
+SELECT to_regcollation('pg_catalog."POSIX"');
+ to_regcollation
+-----------------
+ "POSIX"
+(1 row)
+
-- schemaname not applicable
SELECT regrole('regress_regrole_test');
regrole
@@ -244,6 +268,10 @@ SELECT regtype('ng_catalog.int4');
ERROR: schema "ng_catalog" does not exist
LINE 1: SELECT regtype('ng_catalog.int4');
^
+SELECT regcollation('ng_catalog."POSIX"');
+ERROR: schema "ng_catalog" does not exist
+LINE 1: SELECT regcollation('ng_catalog."POSIX"');
+ ^
-- schemaname not applicable
SELECT regrole('regress_regrole_test');
ERROR: role "regress_regrole_test" does not exist
@@ -315,6 +343,12 @@ SELECT to_regtype('int3');
(1 row)
+SELECT to_regcollation('notacollation');
+ to_regcollation
+-----------------
+
+(1 row)
+
-- with schemaname
SELECT to_regoper('ng_catalog.||/');
to_regoper
@@ -352,6 +386,12 @@ SELECT to_regtype('ng_catalog.int4');
(1 row)
+SELECT to_regcollation('ng_catalog."POSIX"');
+ to_regcollation
+-----------------
+
+(1 row)
+
-- schemaname not applicable
SELECT to_regrole('regress_regrole_test');
to_regrole
diff --git a/src/test/regress/sql/regproc.sql b/src/test/regress/sql/regproc.sql
index a60bc28901..faab0c15ce 100644
--- a/src/test/regress/sql/regproc.sql
+++ b/src/test/regress/sql/regproc.sql
@@ -14,6 +14,7 @@ SELECT regproc('now');
SELECT regprocedure('abs(numeric)');
SELECT regclass('pg_class');
SELECT regtype('int4');
+SELECT regcollation('"POSIX"');
SELECT to_regoper('||/');
SELECT to_regoperator('+(int4,int4)');
@@ -21,6 +22,7 @@ SELECT to_regproc('now');
SELECT to_regprocedure('abs(numeric)');
SELECT to_regclass('pg_class');
SELECT to_regtype('int4');
+SELECT to_regcollation('"POSIX"');
-- with schemaname
@@ -30,12 +32,14 @@ SELECT regproc('pg_catalog.now');
SELECT regprocedure('pg_catalog.abs(numeric)');
SELECT regclass('pg_catalog.pg_class');
SELECT regtype('pg_catalog.int4');
+SELECT regcollation('pg_catalog."POSIX"');
SELECT to_regoper('pg_catalog.||/');
SELECT to_regproc('pg_catalog.now');
SELECT to_regprocedure('pg_catalog.abs(numeric)');
SELECT to_regclass('pg_catalog.pg_class');
SELECT to_regtype('pg_catalog.int4');
+SELECT to_regcollation('pg_catalog."POSIX"');
-- schemaname not applicable
@@ -70,6 +74,7 @@ SELECT regproc('ng_catalog.now');
SELECT regprocedure('ng_catalog.abs(numeric)');
SELECT regclass('ng_catalog.pg_class');
SELECT regtype('ng_catalog.int4');
+SELECT regcollation('ng_catalog."POSIX"');
-- schemaname not applicable
@@ -92,6 +97,7 @@ SELECT to_regproc('know');
SELECT to_regprocedure('absinthe(numeric)');
SELECT to_regclass('pg_classes');
SELECT to_regtype('int3');
+SELECT to_regcollation('notacollation');
-- with schemaname
@@ -101,6 +107,7 @@ SELECT to_regproc('ng_catalog.now');
SELECT to_regprocedure('ng_catalog.abs(numeric)');
SELECT to_regclass('ng_catalog.pg_class');
SELECT to_regtype('ng_catalog.int4');
+SELECT to_regcollation('ng_catalog."POSIX"');
-- schemaname not applicable
--
2.20.1
v17-0004-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From d7681703077afedb4fe8984c8832a5e37731a941 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v17 4/7] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 194 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 132 +++++++++++-
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/indexcmds.c | 11 +
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 19 +-
src/include/catalog/index.h | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 157 ++++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++++
26 files changed, 1048 insertions(+), 68 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b84502295e..f919561923 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21123,7 +21123,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 281f5b7c28..ddcfb7a856 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1670,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1605,8 +1688,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1633,12 +1716,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1694,8 +1783,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1716,8 +1805,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1739,8 +1828,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1773,6 +1867,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1798,10 +1935,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1891,7 +2030,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1979,7 +2119,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2010,7 +2151,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2023,7 +2165,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2036,7 +2179,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2125,7 +2269,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2270,7 +2415,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2292,7 +2439,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2688,8 +2837,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..b756af35d0 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..11a998d3d4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1206,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1301,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2795,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3794,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 84a0b99311..a17bbe4442 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,12 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +49,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +98,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +117,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -545,6 +616,55 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 4e8263af4b..8f42da2444 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3323,6 +3323,17 @@ ReindexRelationConcurrently(Oid relationOid, int options)
/* Start a new transaction to finish process properly */
StartTransactionCommand();
+ /*
+ * Record the current versions of all depended-on collations, for all new
+ * indexes.
+ */
+ foreach(lc, newIndexIds)
+ {
+ Oid indOid = lfirst_oid(lc);
+
+ index_update_collation_versions(indOid);
+ }
+
/* Log what we did */
if (options & REINDEXOPT_VERBOSE)
{
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..8f92601865 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..28ef63d6ab 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..fc3a41221f 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 62e86a19b6..e39e362524 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,17 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..0899a203c0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +133,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+void index_update_collation_versions(Oid relid);
+
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
char relpersistence, int options);
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..82005e9eba 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v17-0005-Preserve-index-dependencies-on-collation-during-.patchtext/plain; charset=us-asciiDownload
From 0986575dfcc68dec2f5b76cb8acffa4bcbf1d7dd Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH v17 5/7] Preserve index dependencies on collation during
pg_upgrade.
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for collations. pg_dump will call this
function to preserve the version information for all indexes, if run with the
--binary-upgrade option.
When pg_upgrade is used to upgrade from a version of PostgreSQL that didn't
support per-object versions, the version will be recorded as unknown. If you
believe that the collation definitions haven't changed since indexes were last
built, you can use --collation-binary-compatible to tell pg_upgrade to assume
that the current versions are compatible with the existing indexes, and it will
pass --unknown-collations-binary-compatible through to pg_dump and mark them as
having the current version (according to the collation provider).
Author: Julien Rouhaud
Reviewed-by: Peter Eisentraut, Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 509 insertions(+), 69 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 5e396f41fb..7d4dbffe5e 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 11a998d3d4..ad682f3395 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3159,6 +3159,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2c6b1bca62..e012258470 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +388,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -704,6 +708,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6907,7 +6915,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6943,7 +6953,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6968,7 +7033,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7007,7 +7074,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7042,7 +7111,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7073,7 +7144,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7107,7 +7180,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7147,6 +7222,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7172,6 +7249,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16456,7 +16535,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16523,6 +16603,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16551,6 +16635,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18527,6 +18626,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0c6444ef6..e3369451a7 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -365,6 +365,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 4ef2036ecd..be2d137376 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index e39e362524..663bc11c33 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 0899a203c0..9753b0cde2 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c7c25e1eae..e403107946 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10194,6 +10194,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
v17-0006-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From 287e6fa273df997df2d65b70bbc8ac585304b9cb Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v17 6/7] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..744789b1bb 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index ad682f3395..43278d8f8a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3159,7 +3159,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8c33b67c1b..3ab12a0ec2 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -554,6 +555,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3872,6 +3874,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4039,6 +4045,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4605,6 +4617,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17260,3 +17277,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7caf0f2f53..6c0a6a2732 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3175,6 +3175,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 804cbafda4..89fe0b38f8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2570,6 +2570,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ae35fa4aa9..43d2524cf1 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1705,7 +1720,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1751,6 +1766,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9753b0cde2..9709be23d0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 079fe1a5f3..64b3e40b70 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v17-0007-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 7b32cc1f993d7632111ccea6bc8615e76fd6c986 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v17 7/7] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 20cdfabd7b..d4aa300c5a 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -936,6 +936,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</para>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On 2020-03-17 18:43, Julien Rouhaud wrote:
On Tue, Mar 17, 2020 at 05:31:47PM +0100, Christoph Berg wrote:
Re: Peter Eisentraut 2020-03-17<fd8d4475-85ad-506f-2dda-f4d6e66785bc@2ndquadrant.com>
Did we discuss the regcollation type? In the current patch set, it's only
used in two places in a new regression test, where it can easily be replaced
by a join. Do we need it?I originally wrote it for a previous version of the patchset, to shorten the
pg_dump query, but that went out when I replaced the DDL command with native
functions instead. It didn't seem to hurt to keep it, so I relied on it in the
regression tests.
OK, I have committed the regcollation patch, and some surrounding
cleanup of the reg* types documentation.
Note that your patch updated the pg_upgrade documentation to say that
tables with regcollation columns cannot be upgraded but didn't actually
patch the pg_upgrade code to make that happen.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On Wed, Mar 18, 2020 at 09:29:55PM +0100, Peter Eisentraut wrote:
On 2020-03-17 18:43, Julien Rouhaud wrote:
On Tue, Mar 17, 2020 at 05:31:47PM +0100, Christoph Berg wrote:
Re: Peter Eisentraut 2020-03-17<fd8d4475-85ad-506f-2dda-f4d6e66785bc@2ndquadrant.com>
Did we discuss the regcollation type? In the current patch set, it's only
used in two places in a new regression test, where it can easily be replaced
by a join. Do we need it?I originally wrote it for a previous version of the patchset, to shorten the
pg_dump query, but that went out when I replaced the DDL command with native
functions instead. It didn't seem to hurt to keep it, so I relied on it in the
regression tests.OK, I have committed the regcollation patch, and some surrounding cleanup of
the reg* types documentation.
Thanks!
Note that your patch updated the pg_upgrade documentation to say that tables
with regcollation columns cannot be upgraded but didn't actually patch the
pg_upgrade code to make that happen.
Oh right, sorry for that I shouldn't have miss it:(
Re: Collation versioning
On Wed, Mar 18, 2020 at 09:29:55PM +0100, Peter Eisentraut wrote:
OK, I have committed the regcollation patch, and some surrounding cleanup of
the reg* types documentation.
Thanks, Peter.
--
Michael
Re: Collation versioning
On Wed, Mar 18, 2020 at 04:35:43PM +0100, Julien Rouhaud wrote:
On Wed, Mar 18, 2020 at 09:56:40AM +0100, Julien Rouhaud wrote:
AFAICT it was only missing a call to index_update_collation_versions() in
ReindexRelationConcurrently. I added regression tests to make sure that
REINDEX, REINDEX [INDEX|TABLE] CONCURRENTLY and VACUUM FULL are doing what's
expected.
If you add a call to index_update_collation_versions(), the old and
invalid index will use the same refobjversion as the new index, which
is the latest collation version of the system, no? If the operation
is interrupted before the invalid index is dropped, then we would keep
a confusing value for refobjversion, because the old invalid index
does not rely on the new collation version, but on the old one.
Hence, it seems to me that it would be correct to have the old invalid
index either use an empty version string to say "we don't know"
because the index is invalid anyway, or keep a reference to the old
collation version intact. I think that the latter is much more useful
for debugging issues when upgrading a subset of indexes if the
operation is interrupted for a reason or another.
Given discussion in nearby threads, I obviously can't add tests for failed
REINDEX CONCURRENTLY, so here's what's happening with a manual repro:=# UPDATE pg_depend SET refobjversion = 'meh' WHERE refobjversion = '153.97';
UPDATE 1
Updates to catalogs are not an existing practice in the core
regression tests, so patches had better not do that. :p
=# REINDEX TABLE CONCURRENTLY t1 ;
LOCATION: ReindexRelationConcurrently, indexcmds.c:2839
^CCancel request sent
ERROR: 57014: canceling statement due to user request
LOCATION: ProcessInterrupts, postgres.c:3171
I guess that you used a second session here beginning a transaction
before REINDEX CONCURRENTLY ran here so as it would stop after
swapping dependencies, right?
=# SELECT objid::regclass, indisvalid, refobjversion
FROM pg_depend d
JOIN pg_index i ON i.indexrelid = d.objid
WHERE refobjversion IS NOT NULL;
objid | indisvalid | refobjversion
------------------+------------+---------------
t1_val_idx_ccold | f | 153.97
t1_val_idx | t | meh
(2 rows)=# REINDEX TABLE t1;
WARNING: 0A000: cannot reindex invalid index "pg_toast.pg_toast_16418_index_ccold" on TOAST table, skipping
LOCATION: reindex_relation, index.c:3987
REINDEX=# SELECT objid::regclass, indisvalid, refobjversion
FROM pg_depend d
JOIN pg_index i ON i.indexrelid = d.objid
WHERE refobjversion IS NOT NULL;
objid | indisvalid | refobjversion
------------------+------------+---------------
t1_val_idx_ccold | t | 153.97
t1_val_idx | t | 153.97
(2 rows)ISTM that it's working as intended.
After a non-concurrent reindex, this information is right. However
based on the output of your test here, after REINDEX CONCURRENTLY the
information held in refobjversion is incorrect for t1_val_idx_ccold
and t1_val_idx. They should be reversed.
--
Michael
Re: Collation versioning
On Thu, Mar 19, 2020 at 12:31:54PM +0900, Michael Paquier wrote:
On Wed, Mar 18, 2020 at 04:35:43PM +0100, Julien Rouhaud wrote:
On Wed, Mar 18, 2020 at 09:56:40AM +0100, Julien Rouhaud wrote:
AFAICT it was only missing a call to index_update_collation_versions() in
ReindexRelationConcurrently. I added regression tests to make sure that
REINDEX, REINDEX [INDEX|TABLE] CONCURRENTLY and VACUUM FULL are doing what's
expected.If you add a call to index_update_collation_versions(), the old and
invalid index will use the same refobjversion as the new index, which
is the latest collation version of the system, no? If the operation
is interrupted before the invalid index is dropped, then we would keep
a confusing value for refobjversion, because the old invalid index
does not rely on the new collation version, but on the old one.
Hence, it seems to me that it would be correct to have the old invalid
index either use an empty version string to say "we don't know"
because the index is invalid anyway, or keep a reference to the old
collation version intact. I think that the latter is much more useful
for debugging issues when upgrading a subset of indexes if the
operation is interrupted for a reason or another.
Indeed, I confused the _ccold and _ccnew indexes. So, the root cause is phase
4, more precisely the dependency swap in index_concurrently_swap.
A possible fix would be to teach changeDependenciesOf() to preserve the
dependency version. It'd be quite bit costly as this would mean an extra index
search for each dependency row found. We could probably skip the lookup if the
row have a NULL recorded version, as version should either be null or non null
for both objects.
I'm wondering if that's a good time to make changeDependenciesOf and
changeDependenciesOn private, and instead expose a swapDependencies(classid,
obj1, obj2) that would call both, as preserving the version doesn't really
makes sense outside a switch. It's als oa good way to ensure that no CCI is
performed in the middle.
Given discussion in nearby threads, I obviously can't add tests for failed
REINDEX CONCURRENTLY, so here's what's happening with a manual repro:=# UPDATE pg_depend SET refobjversion = 'meh' WHERE refobjversion = '153.97';
UPDATE 1Updates to catalogs are not an existing practice in the core
regression tests, so patches had better not do that. :p
I already heavily relied on that in the previous version of the patchset. The
only possible alternative would be to switch to TAP tests, and constantly
restart the instance in binary upgrade mode to be able to call
binary_upgrade_set_index_coll_version. I'd prefer to avoid that if that's
possible, as it'll make the test way more complex and quite unreadable.
=# REINDEX TABLE CONCURRENTLY t1 ;
LOCATION: ReindexRelationConcurrently, indexcmds.c:2839
^CCancel request sent
ERROR: 57014: canceling statement due to user request
LOCATION: ProcessInterrupts, postgres.c:3171I guess that you used a second session here beginning a transaction
before REINDEX CONCURRENTLY ran here so as it would stop after
swapping dependencies, right?
Yes, sorry for eluding that. I'm using a SELECT FOR UPDATE, same scenario as
the recent issue with TOAST tables with REINDEX CONCURRENTLY.
Re: Collation versioning
On Thu, Mar 19, 2020 at 08:12:47PM +0100, Julien Rouhaud wrote:
On Thu, Mar 19, 2020 at 12:31:54PM +0900, Michael Paquier wrote:
On Wed, Mar 18, 2020 at 04:35:43PM +0100, Julien Rouhaud wrote:
On Wed, Mar 18, 2020 at 09:56:40AM +0100, Julien Rouhaud wrote:
AFAICT it was only missing a call to index_update_collation_versions() in
ReindexRelationConcurrently. I added regression tests to make sure that
REINDEX, REINDEX [INDEX|TABLE] CONCURRENTLY and VACUUM FULL are doing what's
expected.If you add a call to index_update_collation_versions(), the old and
invalid index will use the same refobjversion as the new index, which
is the latest collation version of the system, no? If the operation
is interrupted before the invalid index is dropped, then we would keep
a confusing value for refobjversion, because the old invalid index
does not rely on the new collation version, but on the old one.
Hence, it seems to me that it would be correct to have the old invalid
index either use an empty version string to say "we don't know"
because the index is invalid anyway, or keep a reference to the old
collation version intact. I think that the latter is much more useful
for debugging issues when upgrading a subset of indexes if the
operation is interrupted for a reason or another.Indeed, I confused the _ccold and _ccnew indexes. So, the root cause is phase
4, more precisely the dependency swap in index_concurrently_swap.A possible fix would be to teach changeDependenciesOf() to preserve the
dependency version. It'd be quite bit costly as this would mean an extra index
search for each dependency row found. We could probably skip the lookup if the
row have a NULL recorded version, as version should either be null or non null
for both objects.I'm wondering if that's a good time to make changeDependenciesOf and
changeDependenciesOn private, and instead expose a swapDependencies(classid,
obj1, obj2) that would call both, as preserving the version doesn't really
makes sense outside a switch. It's als oa good way to ensure that no CCI is
performed in the middle.
Hearing no complaints, I implemented that approach in attached v18.
Here's the new behavior for interrupted REINDEX CONCURRENTLY:
# drop table if exists t1;create table t1(id integer, val text); create index on t1(val collate "fr-x-icu");
NOTICE: 00000: table "t1" does not exist, skipping
DROP TABLE
CREATE TABLE
CREATE INDEX
# update pg_depend set refobjversion = 'meh' where refobjversion = '153.97';
UPDATE 1
# select objid::regclass, indisvalid, refobjversion from pg_depend d join pg_index i on i.indexrelid = d.objid where refobjversion is not null;
objid | indisvalid | refobjversion
------------+------------+---------------
t1_val_idx | t | meh
(1 row)
(on another session: begin; select * from t1 for update;)
# reindex table CONCURRENTLY t1;
^CCancel request sent
ERROR: 57014: canceling statement due to user request
# select objid::regclass, indisvalid, refobjversion from pg_depend d join pg_index i on i.indexrelid = d.objid where refobjversion is not null;
objid | indisvalid | refobjversion
------------------+------------+---------------
t1_val_idx_ccold | f | meh
t1_val_idx | t | 153.97
(2 rows)
# reindex table CONCURRENTLY t1;
WARNING: 0A000: cannot reindex invalid index "public.t1_val_idx_ccold" concurrently, skipping
WARNING: XX002: cannot reindex invalid index "pg_toast.pg_toast_16385_index_ccold" concurrently, skipping
REINDEX
# select objid::regclass, indisvalid, refobjversion from pg_depend d join pg_index i on i.indexrelid = d.objid where refobjversion is not null;
objid | indisvalid | refobjversion
------------------+------------+---------------
t1_val_idx_ccold | f | meh
t1_val_idx | t | 153.97
(2 rows)
# reindex table t1;
WARNING: 0A000: cannot reindex invalid index "pg_toast.pg_toast_16385_index_ccold" on TOAST table, skipping
REINDEX
# select objid::regclass, indisvalid, refobjversion from pg_depend d join pg_index i on i.indexrelid = d.objid where refobjversion is not null;
objid | indisvalid | refobjversion
------------------+------------+---------------
t1_val_idx_ccold | t | 153.97
t1_val_idx | t | 153.97
(2 rows)
I also rebased the patchset against master (so removing the regcollation
patch), but no other changes otherwise, so there's still the direct updates on
the catalog in the regressoin tests.
Attachments:
v18-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 6a1bc637be5b686c8ac69799989866f8575e4ef0 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v18 1/6] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 64614b569c..dbfb525069 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 464a48ed6a..f26e245c65 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21122,10 +21122,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..7caf0f2f53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3184,16 +3184,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5184,9 +5174,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 88b912977e..05f694929c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3278,9 +3270,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7e384f956c..804cbafda4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -830,7 +830,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10343,21 +10342,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 64fd3ae18a..60dab33fcb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ced0681ec3..2c6b1bca62 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13557,7 +13557,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13629,7 +13633,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 2039b42449..079fe1a5f3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1870,17 +1870,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v18-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From eec56d0b12898c88fd4e9e25b840b8a920702f21 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v18 2/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 ++++++-
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 61 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index dbfb525069..b2a125de78 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2950,6 +2950,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</entry>
</row>
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3115,7 +3126,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index ffd52c1153..281f5b7c28 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1603,7 +1603,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1690,7 +1692,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1710,7 +1714,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2682,7 +2688,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 596dafe19c..84a0b99311 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ab5e92bdc6..62e86a19b6 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -185,6 +185,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v18-0003-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From de71e7a4a5feb8065f9c7ed577caccc3fd4a5544 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v18 3/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 203 ++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 25 +-
src/include/catalog/index.h | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
25 files changed, 1156 insertions(+), 88 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index f26e245c65..98313e2b05 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21122,7 +21122,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 281f5b7c28..ddcfb7a856 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1670,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1605,8 +1688,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1633,12 +1716,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1694,8 +1783,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1716,8 +1805,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1739,8 +1828,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1773,6 +1867,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1798,10 +1935,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1891,7 +2030,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1979,7 +2119,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2010,7 +2151,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2023,7 +2165,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2036,7 +2179,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2125,7 +2269,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2270,7 +2415,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2292,7 +2439,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2688,8 +2837,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..b756af35d0 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 2d81bc3cbc..a169b707a0 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1206,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1301,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1676,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2790,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3789,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 84a0b99311..7f5375fd14 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -402,16 +479,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -436,13 +530,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -465,7 +595,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -545,6 +675,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..8f92601865 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..28ef63d6ab 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..fc3a41221f 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 62e86a19b6..4d99f0e6da 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,17 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -201,11 +214,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..0899a203c0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +133,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+void index_update_collation_versions(Oid relid);
+
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
char relpersistence, int options);
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..82005e9eba 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v18-0004-Preserve-index-dependencies-on-collation-during-.patchtext/plain; charset=us-asciiDownload
From de77eecd1c572e9121afe304f9ee227dd2b1f50e Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Thu, 7 Nov 2019 07:33:20 +0100
Subject: [PATCH v18 4/6] Preserve index dependencies on collation during
pg_upgrade.
A new binary_upgrade_set_index_coll_version() SQL function is added to override
the recorded dependency version for collations. pg_dump will call this
function to preserve the version information for all indexes, if run with the
--binary-upgrade option.
When pg_upgrade is used to upgrade from a version of PostgreSQL that didn't
support per-object versions, the version will be recorded as unknown. If you
believe that the collation definitions haven't changed since indexes were last
built, you can use --collation-binary-compatible to tell pg_upgrade to assume
that the current versions are compatible with the existing indexes, and it will
pass --unknown-collations-binary-compatible through to pg_dump and mark them as
having the current version (according to the collation provider).
Author: Julien Rouhaud
Reviewed-by: Peter Eisentraut, Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
src/backend/catalog/index.c | 62 ++++++
src/backend/utils/adt/pg_upgrade_support.c | 25 +++
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 7 +
src/include/catalog/index.h | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/perl/PostgresNode.pm | 6 +-
15 files changed, 509 insertions(+), 69 deletions(-)
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 49de1d57ab..d0377d8e76 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index a169b707a0..fe6dfd1ad2 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3154,6 +3154,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2c6b1bca62..e012258470 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +388,7 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -704,6 +708,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -6907,7 +6915,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -6943,7 +6953,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -6968,7 +7033,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7007,7 +7074,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7042,7 +7111,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7073,7 +7144,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7107,7 +7180,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7147,6 +7222,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7172,6 +7249,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16456,7 +16535,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16523,6 +16603,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16551,6 +16635,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18527,6 +18626,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0c6444ef6..e3369451a7 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -365,6 +365,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 4ef2036ecd..be2d137376 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 4d99f0e6da..389e28b13e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 0899a203c0..9753b0cde2 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 87d25d4a4b..2d5d1bf4f4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10231,6 +10231,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 9575268bd7..bba07a2319 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
--
2.20.1
v18-0005-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From e4e474384d584055282fddad43c4c4b1d4252aac Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v18 5/6] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..744789b1bb 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fe6dfd1ad2..38a17128e7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3154,7 +3154,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 729025470d..d213d5992a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -554,6 +555,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3872,6 +3874,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4039,6 +4045,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4605,6 +4617,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17261,3 +17278,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7caf0f2f53..6c0a6a2732 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3175,6 +3175,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 804cbafda4..89fe0b38f8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2570,6 +2570,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ae35fa4aa9..43d2524cf1 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1705,7 +1720,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1751,6 +1766,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9753b0cde2..9709be23d0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 079fe1a5f3..64b3e40b70 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1844,7 +1844,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1860,6 +1861,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v18-0006-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From e785c0176fc94e7a9433b350a0a028daec837db7 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v18 6/6] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 20cdfabd7b..d4aa300c5a 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -936,6 +936,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</para>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Fri, Mar 20, 2020 at 02:52:33PM +0100, Julien Rouhaud wrote:
On Thu, Mar 19, 2020 at 08:12:47PM +0100, Julien Rouhaud wrote:
On Thu, Mar 19, 2020 at 12:31:54PM +0900, Michael Paquier wrote:
On Wed, Mar 18, 2020 at 04:35:43PM +0100, Julien Rouhaud wrote:
On Wed, Mar 18, 2020 at 09:56:40AM +0100, Julien Rouhaud wrote:
AFAICT it was only missing a call to index_update_collation_versions() in
ReindexRelationConcurrently. I added regression tests to make sure that
REINDEX, REINDEX [INDEX|TABLE] CONCURRENTLY and VACUUM FULL are doing what's
expected.If you add a call to index_update_collation_versions(), the old and
invalid index will use the same refobjversion as the new index, which
is the latest collation version of the system, no? If the operation
is interrupted before the invalid index is dropped, then we would keep
a confusing value for refobjversion, because the old invalid index
does not rely on the new collation version, but on the old one.
Hence, it seems to me that it would be correct to have the old invalid
index either use an empty version string to say "we don't know"
because the index is invalid anyway, or keep a reference to the old
collation version intact. I think that the latter is much more useful
for debugging issues when upgrading a subset of indexes if the
operation is interrupted for a reason or another.Indeed, I confused the _ccold and _ccnew indexes. So, the root cause is phase
4, more precisely the dependency swap in index_concurrently_swap.A possible fix would be to teach changeDependenciesOf() to preserve the
dependency version. It'd be quite bit costly as this would mean an extra index
search for each dependency row found. We could probably skip the lookup if the
row have a NULL recorded version, as version should either be null or non null
for both objects.I'm wondering if that's a good time to make changeDependenciesOf and
changeDependenciesOn private, and instead expose a swapDependencies(classid,
obj1, obj2) that would call both, as preserving the version doesn't really
makes sense outside a switch. It's als oa good way to ensure that no CCI is
performed in the middle.Hearing no complaints, I implemented that approach in attached v18.
Here's the new behavior for interrupted REINDEX CONCURRENTLY:
# drop table if exists t1;create table t1(id integer, val text); create index on t1(val collate "fr-x-icu");
NOTICE: 00000: table "t1" does not exist, skipping
DROP TABLE
CREATE TABLE
CREATE INDEX# update pg_depend set refobjversion = 'meh' where refobjversion = '153.97';
UPDATE 1# select objid::regclass, indisvalid, refobjversion from pg_depend d join pg_index i on i.indexrelid = d.objid where refobjversion is not null;
objid | indisvalid | refobjversion
------------+------------+---------------
t1_val_idx | t | meh
(1 row)(on another session: begin; select * from t1 for update;)
# reindex table CONCURRENTLY t1;
^CCancel request sent
ERROR: 57014: canceling statement due to user request# select objid::regclass, indisvalid, refobjversion from pg_depend d join pg_index i on i.indexrelid = d.objid where refobjversion is not null;
objid | indisvalid | refobjversion
------------------+------------+---------------
t1_val_idx_ccold | f | meh
t1_val_idx | t | 153.97
(2 rows)# reindex table CONCURRENTLY t1;
WARNING: 0A000: cannot reindex invalid index "public.t1_val_idx_ccold" concurrently, skipping
WARNING: XX002: cannot reindex invalid index "pg_toast.pg_toast_16385_index_ccold" concurrently, skipping
REINDEX# select objid::regclass, indisvalid, refobjversion from pg_depend d join pg_index i on i.indexrelid = d.objid where refobjversion is not null;
objid | indisvalid | refobjversion
------------------+------------+---------------
t1_val_idx_ccold | f | meh
t1_val_idx | t | 153.97
(2 rows)# reindex table t1;
WARNING: 0A000: cannot reindex invalid index "pg_toast.pg_toast_16385_index_ccold" on TOAST table, skipping
REINDEX# select objid::regclass, indisvalid, refobjversion from pg_depend d join pg_index i on i.indexrelid = d.objid where refobjversion is not null;
objid | indisvalid | refobjversion
------------------+------------+---------------
t1_val_idx_ccold | t | 153.97
t1_val_idx | t | 153.97
(2 rows)I also rebased the patchset against master (so removing the regcollation
patch), but no other changes otherwise, so there's still the direct updates on
the catalog in the regressoin tests.
Conflict since 2f9eb3132 (pg_dump: Allow dumping data of specific foreign
servers), v19 attached.
Attachments:
v19-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 17c6b334a236e4cab81df7302fa79346e102dca4 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v19 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 64614b569c..dbfb525069 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index a329f61f33..822c4405af 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21192,10 +21192,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index c9a90d1191..8e5c433ae6 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3185,16 +3185,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5185,9 +5175,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index d05ca26fcf..921a2a33dc 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1105,14 +1105,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3279,9 +3271,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 87a80bc25c..d6d41ac76e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -245,7 +245,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -832,7 +832,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10355,21 +10354,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2562eb5416..4f42f342b0 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 408637cfec..cd3d6ed60a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13657,7 +13657,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13729,7 +13733,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 77943f0637..c9ea09e4aa 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1871,17 +1871,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v19-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From 267cfcf97a6e7b4df17765ed9d4fc28b1ad08954 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v19 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 ++++++-
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 61 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index dbfb525069..b2a125de78 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2950,6 +2950,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</entry>
</row>
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3115,7 +3126,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index ffd52c1153..281f5b7c28 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1603,7 +1603,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1690,7 +1692,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1710,7 +1714,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2682,7 +2688,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 596dafe19c..84a0b99311 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ab5e92bdc6..62e86a19b6 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -185,6 +185,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v19-0003-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From 0ee158a7fbd0a49e1005e929bd7d733624ea36ed Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v19 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 822c4405af..9c9ab19ec0 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21192,7 +21192,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 997ef1e12a..4b26838ad0 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 281f5b7c28..ddcfb7a856 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1670,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1605,8 +1688,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1633,12 +1716,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1694,8 +1783,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1716,8 +1805,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1739,8 +1828,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1773,6 +1867,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1798,10 +1935,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1891,7 +2030,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1979,7 +2119,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2010,7 +2151,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2023,7 +2165,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2036,7 +2179,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2125,7 +2269,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2270,7 +2415,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2292,7 +2439,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2688,8 +2837,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 632c058b80..0e6c3f50c9 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2306,7 +2306,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2316,7 +2316,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3640,7 +3640,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index bd7ec923e9..74e5099283 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1030,6 +1033,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1120,21 +1127,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1148,21 +1211,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1241,6 +1313,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1688,14 +1848,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2649,6 +2804,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3002,6 +3168,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3637,6 +3865,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 84a0b99311..7f5375fd14 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -402,16 +479,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -436,13 +530,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -465,7 +595,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -545,6 +675,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 3a89f8fe1e..aa4c7b3ec5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 51470dd73e..52f22f2379 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 4f42f342b0..129862c1ca 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1469,7 +1474,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1551,6 +1556,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index f8e2c6e88e..a929cfd8e9 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5739,6 +5740,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index cd3d6ed60a..e0534c3527 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -290,6 +291,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -391,6 +394,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -721,6 +725,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7007,7 +7015,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7043,7 +7053,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7068,7 +7133,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7107,7 +7174,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7142,7 +7211,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7173,7 +7244,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7207,7 +7280,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7247,6 +7322,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7272,6 +7349,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16544,7 +16623,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16611,6 +16691,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16639,6 +16723,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18620,6 +18719,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 3e11166615..f7f33c35c3 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 62e86a19b6..389e28b13e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -156,7 +163,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,17 +184,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -201,11 +221,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..9753b0cde2 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+void index_update_collation_versions(Oid relid);
+
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
char relpersistence, int options);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a649e44d08..d29ae8e3e6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10254,6 +10254,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 74106b3731..b12063306e 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1d5450758e..39f84a3844 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v19-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From c5f1a12cb7c9e9175ff8c777c32e71ec3758a613 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v19 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 26 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 6d34dbb74e..744789b1bb 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -109,6 +110,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 74e5099283..ca4caeb37c 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3168,7 +3168,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index c8c88be2c9..af8eab562c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -554,6 +555,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3872,6 +3874,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4039,6 +4045,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4605,6 +4617,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17263,3 +17280,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 8e5c433ae6..4c93879d30 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3176,6 +3176,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d6d41ac76e..9737860061 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2572,6 +2572,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5fec59723c..bc04a864c1 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1709,7 +1724,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION");
+ "RESET", "ATTACH PARTITION", "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1755,6 +1770,15 @@ psql_completion(const char *text, int start, int end)
"buffering =", /* GiST */
"pages_per_range =", "autosummarize =" /* BRIN */
);
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9753b0cde2..9709be23d0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index c9ea09e4aa..2423b021a7 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1845,7 +1845,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1861,6 +1862,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v19-0005-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From d4a10e3fc19c50b848a0594f89833f624f9360b9 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v19 5/5] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index b6023fa459..a460cb5be8 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -946,6 +946,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Thu, Apr 02, 2020 at 03:00:45PM +0200, Julien Rouhaud wrote:
Conflict since 2f9eb3132 (pg_dump: Allow dumping data of specific foreign
servers), v19 attached.
New rebase due to recent conflicts.
Attachments:
v20-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From f5dbff418d40de5d4161a85eb8b29c7a19441505 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v20 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 5 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index ce33df9e58..6463baf77d 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 606defc515..5cf3c8a836 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -23005,10 +23005,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system. If this is different from the value
- in <literal>pg_collation.collversion</literal>, then objects depending on
- the collation might need to be rebuilt. See also
- <xref linkend="sql-altercollation"/>.
+ operating system.
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 491452ae2d..f094baa781 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5232,9 +5222,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 8408c28ec6..3394554a66 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3284,9 +3276,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 3c78f2d1b5..0ae146be02 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -841,7 +841,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10378,21 +10377,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2562eb5416..4f42f342b0 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 5db4f5761d..4854b9d4d5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13678,7 +13678,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13750,7 +13754,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 5e1ffafb91..7a23fb7529 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1873,17 +1873,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v20-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From f4fa056cc5fbd8c2bf511fcca1c19c8c3c1e4af9 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v20 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 ++++++-
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 61 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 6463baf77d..b5106acad0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2950,6 +2950,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</entry>
</row>
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3115,7 +3126,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index ffd52c1153..281f5b7c28 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1603,7 +1603,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1690,7 +1692,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1710,7 +1714,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2682,7 +2688,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index fa38ee9477..26a7f1d216 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 2c6abe26a5..0ad0597050 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -185,6 +185,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v20-0003-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From 7e588fe1b4560f251c372fa4c55c816faf819147 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v20 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5cf3c8a836..4d65dbf6dd 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -23005,7 +23005,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 997ef1e12a..4b26838ad0 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 281f5b7c28..ddcfb7a856 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1670,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1605,8 +1688,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1633,12 +1716,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1694,8 +1783,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1716,8 +1805,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1739,8 +1828,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1773,6 +1867,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1798,10 +1935,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1891,7 +2030,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1979,7 +2119,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2010,7 +2151,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2023,7 +2165,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2036,7 +2179,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2125,7 +2269,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2270,7 +2415,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2292,7 +2439,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2688,8 +2837,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 632c058b80..0e6c3f50c9 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2306,7 +2306,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2316,7 +2316,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3640,7 +3640,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7cfbdd57db..c8d643a3ed 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1687,14 +1847,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2648,6 +2803,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3001,6 +3167,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3629,6 +3857,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 26a7f1d216..e118e041c1 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 5a110edb07..57e0b6e900 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 4f42f342b0..129862c1ca 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1469,7 +1474,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1551,6 +1556,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9f1f11d0c1..e40e794b11 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5925,6 +5926,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 4854b9d4d5..02de380900 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -290,6 +291,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -391,6 +394,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -721,6 +725,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7023,7 +7031,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7059,7 +7069,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7084,7 +7149,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7123,7 +7190,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7158,7 +7227,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7189,7 +7260,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7223,7 +7296,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7263,6 +7338,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7288,6 +7365,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16565,7 +16644,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16632,6 +16712,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16660,6 +16744,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18641,6 +18740,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 61c909e06d..62a8e12c1b 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0ad0597050..42a4583be1 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -156,7 +163,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,17 +184,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -205,11 +225,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4bce3ad8de..9e492217aa 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10333,6 +10333,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1d5450758e..39f84a3844 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v20-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From 6db4d4c9140b268d124354d649bdc210bc449a14 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v20 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 136 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index de6f89d458..957a183677 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c8d643a3ed..0d80f4a842 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3167,7 +3167,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 5745cd648a..157dd9e214 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -556,6 +557,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3874,6 +3876,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4041,6 +4047,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4607,6 +4619,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17379,3 +17396,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index f094baa781..44d85e7744 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0ae146be02..31119f4a07 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2581,6 +2581,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index f6fd623c98..d6e3323904 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1709,7 +1724,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1759,6 +1775,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7a23fb7529..423f33b72c 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1847,7 +1847,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1863,6 +1864,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v20-0005-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 7d6bb1fb05764e2432b55cb859774504b635e371 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v20 5/5] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index b6023fa459..a460cb5be8 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -946,6 +946,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Fri, Apr 24, 2020 at 4:49 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Thu, Apr 02, 2020 at 03:00:45PM +0200, Julien Rouhaud wrote:
Conflict since 2f9eb3132 (pg_dump: Allow dumping data of specific foreign
servers), v19 attached.New rebase due to recent conflicts.
New conflict, v21 attached.
Attachments:
v21-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchapplication/octet-stream; name=v21-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchDownload
From 8d8f555cc388bef7a82599b33c7a254c914dfece Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v21 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 136 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index de6f89d458..957a183677 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c8d643a3ed..0d80f4a842 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3167,7 +3167,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 5745cd648a..157dd9e214 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -556,6 +557,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3874,6 +3876,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4041,6 +4047,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4607,6 +4619,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17379,3 +17396,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index f094baa781..44d85e7744 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0ae146be02..31119f4a07 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2581,6 +2581,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index f6fd623c98..d6e3323904 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1709,7 +1724,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1759,6 +1775,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7a23fb7529..423f33b72c 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1847,7 +1847,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1863,6 +1864,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v21-0003-Track-collation-versions-for-indexes.patchapplication/octet-stream; name=v21-0003-Track-collation-versions-for-indexes.patchDownload
From 8e5b1b64eaf2509f07f45e612891df94835db084 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v21 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 686b1221fd..e67d7b188d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25388,7 +25388,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 997ef1e12a..4b26838ad0 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 281f5b7c28..ddcfb7a856 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1670,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1605,8 +1688,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1633,12 +1716,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1694,8 +1783,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1716,8 +1805,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1739,8 +1828,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1773,6 +1867,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1798,10 +1935,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1891,7 +2030,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1979,7 +2119,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2010,7 +2151,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2023,7 +2165,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2036,7 +2179,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2125,7 +2269,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2270,7 +2415,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2292,7 +2439,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2688,8 +2837,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 632c058b80..0e6c3f50c9 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2306,7 +2306,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2316,7 +2316,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3640,7 +3640,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7cfbdd57db..c8d643a3ed 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1687,14 +1847,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2648,6 +2803,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3001,6 +3167,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3629,6 +3857,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 26a7f1d216..e118e041c1 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 5a110edb07..57e0b6e900 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 4f42f342b0..129862c1ca 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1469,7 +1474,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1551,6 +1556,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9f1f11d0c1..e40e794b11 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5925,6 +5926,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 4854b9d4d5..02de380900 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -290,6 +291,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -391,6 +394,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -721,6 +725,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7023,7 +7031,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7059,7 +7069,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7084,7 +7149,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7123,7 +7190,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7158,7 +7227,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7189,7 +7260,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7223,7 +7296,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7263,6 +7338,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7288,6 +7365,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16565,7 +16644,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16632,6 +16712,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16660,6 +16744,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18641,6 +18740,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 61c909e06d..62a8e12c1b 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0ad0597050..42a4583be1 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -156,7 +163,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,17 +184,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -205,11 +225,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4bce3ad8de..9e492217aa 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10333,6 +10333,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1d5450758e..39f84a3844 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v21-0001-Remove-pg_collation.collversion.patchapplication/octet-stream; name=v21-0001-Remove-pg_collation.collversion.patchDownload
From 6f4b547e0147fca8c1e20762324cbaae45aec301 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v21 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 65 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 322 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index ce33df9e58..6463baf77d 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2122,17 +2122,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<entry></entry>
<entry><symbol>LC_CTYPE</symbol> for this collation object</entry>
</row>
-
- <row>
- <entry><structfield>collversion</structfield></entry>
- <entry><type>text</type></entry>
- <entry></entry>
- <entry>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d9b3598977..686b1221fd 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25388,11 +25388,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 4cfcb42251..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,72 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"
- endterm="sql-altercollation-notes-title"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes">
- <title id="sql-altercollation-notes-title">Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index def4dda6e8..36120385d1 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -24,7 +24,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -146,26 +145,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 491452ae2d..f094baa781 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5232,9 +5222,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 8408c28ec6..3394554a66 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3284,9 +3276,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 3c78f2d1b5..0ae146be02 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -841,7 +841,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10378,21 +10377,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b1f7f6e2d0..9cecf409a4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1793,10 +1793,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2944,10 +2940,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3560,10 +3552,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2562eb5416..4f42f342b0 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1352,8 +1352,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1455,41 +1453,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 5db4f5761d..4854b9d4d5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13678,7 +13678,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13750,7 +13754,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 5e1ffafb91..7a23fb7529 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1873,17 +1873,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v21-0002-Add-pg_depend.refobjversion.patchapplication/octet-stream; name=v21-0002-Add-pg_depend.refobjversion.patchDownload
From 2db70054e627709c7685a2d749a216fcb42df0dc Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v21 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 ++++++-
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 61 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 6463baf77d..b5106acad0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2950,6 +2950,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
</entry>
</row>
+ <row>
+ <entry><structfield>refobjversion</structfield></entry>
+ <entry><type>text</type></entry>
+ <entry></entry>
+ <entry>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3115,7 +3126,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index ffd52c1153..281f5b7c28 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1603,7 +1603,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1690,7 +1692,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1710,7 +1714,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2682,7 +2688,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index fa38ee9477..26a7f1d216 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index a6577486ce..b7e01a4678 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 2c6abe26a5..0ad0597050 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -185,6 +185,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v21-0005-doc-Add-Collation-Versions-section.patchapplication/octet-stream; name=v21-0005-doc-Add-Collation-Versions-section.patchDownload
From 6e29333fafef72ec84187cfcfcf7b0b7ceff7bc5 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v21 5/5] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 04e71f17d3..fae8dc1d52 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Tue, May 5, 2020 at 5:14 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Fri, Apr 24, 2020 at 4:49 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Thu, Apr 02, 2020 at 03:00:45PM +0200, Julien Rouhaud wrote:
Conflict since 2f9eb3132 (pg_dump: Allow dumping data of specific foreign
servers), v19 attached.New rebase due to recent conflicts.
New conflict, v21 attached.
New conflict, v22 attached.
Attachments:
v22-0005-doc-Add-Collation-Versions-section.patchapplication/octet-stream; name=v22-0005-doc-Add-Collation-Versions-section.patchDownload
From 3407a1d9dd93f91ca2651b5906de91d4486c9d01 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v22 5/5] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 4b4563c5b9..c537bdfc28 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
v22-0001-Remove-pg_collation.collversion.patchapplication/octet-stream; name=v22-0001-Remove-pg_collation.collversion.patchDownload
From 85cf94b5a774b52c6ce829d4dde026f3f87ef075 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v22 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 64 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 321 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 9851ef2713..4ff89ceb9d 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2352,17 +2352,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7c06afd3ea..7fbbb36454 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25388,11 +25388,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index 56f94627c6..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,71 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting>
- </para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 8559779a4f..c78192e34b 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 491452ae2d..f094baa781 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5232,9 +5222,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 8408c28ec6..3394554a66 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3284,9 +3276,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index e66b850e1a..11ba16bf9f 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -840,7 +840,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10373,21 +10372,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 97cbaa3072..7453802fce 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1795,10 +1795,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2946,10 +2942,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3562,10 +3554,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 11d05c73ac..2c5f5bfa37 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1522,8 +1522,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1625,41 +1623,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a4e949c636..1a9fde1445 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13672,7 +13672,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13744,7 +13748,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 6955bb1273..cfde555366 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
extern void RemoveCollationById(Oid collationOid);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 5e1ffafb91..7a23fb7529 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1873,17 +1873,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v22-0002-Add-pg_depend.refobjversion.patchapplication/octet-stream; name=v22-0002-Add-pg_depend.refobjversion.patchDownload
From c6f0340c8002a778aaaa630a4149c21b1f95dec5 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v22 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 13 ++++++-
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 62 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 4ff89ceb9d..0c13e6ff6a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3292,6 +3292,18 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </para></entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3457,7 +3469,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 5565e6fc19..23e016fb5e 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1606,7 +1606,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1693,7 +1695,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1713,7 +1717,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2685,7 +2691,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 21cfdcace9..5d45e706cb 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 67021a6dc1..c9bdc1d5ac 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1560,55 +1560,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v22-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchapplication/octet-stream; name=v22-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchDownload
From dc662a4aa5e7ffef204195ace5f6d48cd6e544a6 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v22 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 136 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index de6f89d458..957a183677 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c8d643a3ed..0d80f4a842 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3167,7 +3167,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8801af589c..b7e66ac2e4 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -556,6 +557,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3926,6 +3928,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4093,6 +4099,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4659,6 +4671,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17478,3 +17495,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index f094baa781..44d85e7744 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 11ba16bf9f..5fb90a47ec 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2580,6 +2580,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index eb018854a5..ec0ef1feae 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1709,7 +1724,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1759,6 +1775,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7a23fb7529..423f33b72c 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1847,7 +1847,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1863,6 +1864,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v22-0003-Track-collation-versions-for-indexes.patchapplication/octet-stream; name=v22-0003-Track-collation-versions-for-indexes.patchDownload
From 2d6a7c7a093a350ac8fc9edc492a112ae9887765 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v22 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7fbbb36454..c96dbfb05d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25388,7 +25388,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 905167690b..01ac326626 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 23e016fb5e..5d1c059927 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -436,6 +440,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1594,6 +1673,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1608,8 +1691,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1636,12 +1719,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1697,8 +1786,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1719,8 +1808,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1742,8 +1831,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1776,6 +1870,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1801,10 +1938,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1894,7 +2033,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1982,7 +2122,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2013,7 +2154,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2026,7 +2168,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2039,7 +2182,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2128,7 +2272,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2273,7 +2418,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2295,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2691,8 +2840,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e393c93a45..79528bc589 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2306,7 +2306,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2316,7 +2316,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3640,7 +3640,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7cfbdd57db..c8d643a3ed 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1687,14 +1847,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2648,6 +2803,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3001,6 +3167,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3629,6 +3857,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 5a110edb07..57e0b6e900 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1a9fde1445..803c79c1d8 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16559,7 +16638,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16626,6 +16706,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16654,6 +16738,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18635,6 +18734,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 61f2c2f5b4..12363134aa 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10331,6 +10331,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
Re: Collation versioning
On Sat, May 23, 2020 at 08:58:45PM +0200, Julien Rouhaud wrote:
On Tue, May 5, 2020 at 5:14 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Fri, Apr 24, 2020 at 4:49 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Thu, Apr 02, 2020 at 03:00:45PM +0200, Julien Rouhaud wrote:
Conflict since 2f9eb3132 (pg_dump: Allow dumping data of specific foreign
servers), v19 attached.New rebase due to recent conflicts.
New conflict, v21 attached.
New conflict, v22 attached.
Recent conflict on the documentation, v23 attached.
Attachments:
v23-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 07d29573e5c022732732e4db77efcb37f8b3b37a Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v23 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 320 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 700271fd40..de28bec87f 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2352,17 +2352,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index a8d57f4e39..dd1c1e70f7 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25385,11 +25385,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index bee6f0dd3c..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index d8cf87e6d0..5cbbe3ba2e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5232,9 +5222,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 627b026b19..73a9507b80 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3284,9 +3276,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9ed640c379..bab489163b 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -840,7 +840,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10373,21 +10372,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 97cbaa3072..7453802fce 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1795,10 +1795,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2946,10 +2942,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3562,10 +3554,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 11d05c73ac..2c5f5bfa37 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1522,8 +1522,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1625,41 +1623,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dfe43968b8..aec1f435b0 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13672,7 +13672,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13744,7 +13748,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 5e1ffafb91..7a23fb7529 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1873,17 +1873,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v23-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From a53d1d84d46b72fab6b46ccf10a62a0ffbac7116 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v23 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 13 ++++++-
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 62 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index de28bec87f..ebe12affe7 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3292,6 +3292,18 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </para></entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3457,7 +3469,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index b33a2f94af..070b27c692 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1598,7 +1598,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1685,7 +1687,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1705,7 +1709,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2677,7 +2683,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 21cfdcace9..5d45e706cb 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 786672b1b6..4b2a58cf18 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1572,55 +1572,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v23-0003-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From 95d55680078a20f43e95c744e5dacc83bf911088 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v23 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index dd1c1e70f7..8413db4adc 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25385,7 +25385,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 319d613296..bfa6f012f3 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e393c93a45..79528bc589 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2306,7 +2306,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2316,7 +2316,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3640,7 +3640,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index aec1f435b0..6897a1f80e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16559,7 +16638,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16626,6 +16706,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16654,6 +16738,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18635,6 +18734,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 61f2c2f5b4..12363134aa 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10331,6 +10331,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v23-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From e90d7681e8a3ca8bf6e13dfe379fd9e4a0280390 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v23 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 136 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index de6f89d458..957a183677 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 210b25d147..34dbae05a2 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3171,7 +3171,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2ab02e01a0..009b6e6906 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -556,6 +557,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3926,6 +3928,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4093,6 +4099,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4659,6 +4671,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17479,3 +17496,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 5cbbe3ba2e..ebf9118f4f 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index bab489163b..13be2b54aa 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2580,6 +2580,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index eb018854a5..ec0ef1feae 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1709,7 +1724,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1759,6 +1775,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7a23fb7529..423f33b72c 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1847,7 +1847,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1863,6 +1864,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v23-0005-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 00435fca9e2179ecc408949842a1ea4e5056629b Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v23 5/5] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 4b4563c5b9..c537bdfc28 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
vOn Thu, Jun 11, 2020 at 08:42:46AM +0200, Julien Rouhaud wrote:
On Sat, May 23, 2020 at 08:58:45PM +0200, Julien Rouhaud wrote:
On Tue, May 5, 2020 at 5:14 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Fri, Apr 24, 2020 at 4:49 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Thu, Apr 02, 2020 at 03:00:45PM +0200, Julien Rouhaud wrote:
Conflict since 2f9eb3132 (pg_dump: Allow dumping data of specific foreign
servers), v19 attached.New rebase due to recent conflicts.
New conflict, v21 attached.
New conflict, v22 attached.
Recent conflict on the documentation, v23 attached.
Same, v24 attached.
Attachments:
v24-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 1c896fbcbe7bd72a3efde84fac9e47ca1571aeeb Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v24 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 320 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 49a881b262..57f3dece43 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2352,17 +2352,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index f065856535..69fe242761 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,11 +25419,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index bee6f0dd3c..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index d8cf87e6d0..5cbbe3ba2e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5232,9 +5222,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 627b026b19..73a9507b80 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3284,9 +3276,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4ff35095b8..c43b16037e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -838,7 +838,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10137,21 +10136,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 97cbaa3072..7453802fce 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1795,10 +1795,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2946,10 +2942,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3562,10 +3554,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 11d05c73ac..2c5f5bfa37 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1522,8 +1522,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1625,41 +1623,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a41a3db876..2f6bfede8e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13672,7 +13672,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13744,7 +13748,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 5e1ffafb91..7a23fb7529 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1873,17 +1873,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v24-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From 1aeba6bff535fa946ac47eea56fab5f5409b275b Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v24 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 13 ++++++-
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 62 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 57f3dece43..7299fc89ef 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3292,6 +3292,18 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </para></entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3457,7 +3469,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index b33a2f94af..070b27c692 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1598,7 +1598,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1685,7 +1687,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1705,7 +1709,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2677,7 +2683,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 21cfdcace9..5d45e706cb 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 786672b1b6..4b2a58cf18 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1572,55 +1572,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v24-0003-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From 3e130e2ce338134eaf1d17645280950885199f8c Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v24 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From 97b3fec0a68c735994b1ec997cc9311a576d4864 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v24 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 136 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index a5e3b06ee4..03355164b3 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 210b25d147..34dbae05a2 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3171,7 +3171,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f79044f39f..cdf3cbf5b1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -556,6 +557,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3926,6 +3928,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4093,6 +4099,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4659,6 +4671,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17477,3 +17494,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 5cbbe3ba2e..ebf9118f4f 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c43b16037e..68934bfeb1 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2594,6 +2594,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index eb018854a5..ec0ef1feae 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1709,7 +1724,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1759,6 +1775,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7a23fb7529..423f33b72c 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1847,7 +1847,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1863,6 +1864,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v24-0005-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From f3785336d115ba3e4556739696a7d1a8ea5eb46b Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v24 5/5] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 4b4563c5b9..c537bdfc28 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Wed, Jul 01, 2020 at 10:52:37AM +0200, Julien Rouhaud wrote:
vOn Thu, Jun 11, 2020 at 08:42:46AM +0200, Julien Rouhaud wrote:
On Sat, May 23, 2020 at 08:58:45PM +0200, Julien Rouhaud wrote:
On Tue, May 5, 2020 at 5:14 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Fri, Apr 24, 2020 at 4:49 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Thu, Apr 02, 2020 at 03:00:45PM +0200, Julien Rouhaud wrote:
Conflict since 2f9eb3132 (pg_dump: Allow dumping data of specific foreign
servers), v19 attached.New rebase due to recent conflicts.
New conflict, v21 attached.
New conflict, v22 attached.
Recent conflict on the documentation, v23 attached.
Same, v24 attached.
And the recent ObjectAddress refactoring just broke the patchset again, so v25
attached.
Attachments:
v25-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From b713594afb14758e6de9f2a6046014f863f859f9 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v25 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 --------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 85 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 8 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 11 insertions(+), 320 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 49a881b262..57f3dece43 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2352,17 +2352,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index f065856535..69fe242761 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,11 +25419,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index bee6f0dd3c..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..78eceda848 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -68,7 +68,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -166,9 +165,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +211,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +219,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +269,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +526,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +586,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +647,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index d8cf87e6d0..5cbbe3ba2e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5232,9 +5222,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 627b026b19..73a9507b80 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3284,9 +3276,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4ff35095b8..c43b16037e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -838,7 +838,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10137,21 +10136,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 97cbaa3072..7453802fce 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1795,10 +1795,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2946,10 +2942,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3562,10 +3554,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 11d05c73ac..2c5f5bfa37 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1522,8 +1522,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1625,41 +1623,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a41a3db876..2f6bfede8e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13672,7 +13672,11 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
/* Get collation-specific details */
appendPQExpBufferStr(query, "SELECT ");
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 130000)
+ appendPQExpBufferStr(query,
+ "collprovider, "
+ "NULL AS collversion, ");
+ else if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
"collprovider, "
"collversion, ");
@@ -13744,7 +13748,7 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
* For binary upgrade, carry over the collation version. For normal
* dump/restore, omit the version, so that it is computed upon restore.
*/
- if (dopt->binary_upgrade)
+ if (dopt->binary_upgrade && fout->remoteVersion < 130000)
{
int i_collversion;
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 5e1ffafb91..7a23fb7529 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1873,17 +1873,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v25-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From f224e8ed54616cf0e1e3bc6a2127a97f6ad91e09 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v25 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and perhaps more things later.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud, Peter Eisentraut and Michael Paquier
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 13 ++++++-
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 62 insertions(+), 33 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 57f3dece43..7299fc89ef 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3292,6 +3292,18 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </para></entry>
+ </row>
+
</tbody>
</tgroup>
</table>
@@ -3457,7 +3469,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
dependencies' restrictions about which objects must be dropped together
must be satisfied.
</para>
-
</sect1>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index b33a2f94af..070b27c692 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1598,7 +1598,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1685,7 +1687,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1705,7 +1709,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2677,7 +2683,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 21cfdcace9..5d45e706cb 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 786672b1b6..4b2a58cf18 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1572,55 +1572,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v25-0003-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From dddbd4deb4f131a6e1a5264f680ff96b1d9978c4 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v25 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From 2b9e849516b7506ccaecee216b7670129818f94b Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v25 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
This command allows privileged users to specify that the currently installed
collation version, for a specific collation, is binary compatible with the one
that was installed when the specified index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud
Reviewed-by: Laurenz Albe, Thomas Munro and Peter Eisentraut
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 46 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 136 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index a5e3b06ee4..03355164b3 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e85aeb5502..ef2b896b6f 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,7 +3156,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f79044f39f..cdf3cbf5b1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -556,6 +557,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3926,6 +3928,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4093,6 +4099,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4659,6 +4671,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17477,3 +17494,32 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/* Execute an ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This override an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 5cbbe3ba2e..ebf9118f4f 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c43b16037e..68934bfeb1 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2594,6 +2594,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index eb018854a5..ec0ef1feae 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1709,7 +1724,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1759,6 +1775,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7a23fb7529..423f33b72c 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1847,7 +1847,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1863,6 +1864,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v25-0005-doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 6e4b40b889b356f0547afe1f64dd52ef64a205c6 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v25 5/5] doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro
Reviewed-by: Julien Rouhaud
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 4b4563c5b9..c537bdfc28 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be reported
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Thu, Jul 2, 2020 at 3:03 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
And the recent ObjectAddress refactoring just broke the patchset again, so v25
attached.
I found some more things to remove from pg_dump.c and collationcmds.c
relating pg_collation.collversion. I also updated a couple of
mentions of release 13 in the code and docs, and made some minor
language improvements here and there.
I still wish I had a better idea than this:
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}I'm doing some more testing and looking for weird cases... More soon.
Attachments:
v26-0001-Remove-pg_collation.collversion.patchtext/x-patch; charset=US-ASCII; name=v26-0001-Remove-pg_collation.collversion.patchDownload
From e19b650eb95e703f21de35e7d0fd70634b7b0aa0 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v26 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 -------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 88 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 24 +----
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 7 insertions(+), 343 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index e9cdff4864..04fcb5fd33 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2353,17 +2353,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index f065856535..69fe242761 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,11 +25419,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index bee6f0dd3c..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..5ad8886e60 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -61,14 +61,12 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *lcctypeEl = NULL;
DefElem *providerEl = NULL;
DefElem *deterministicEl = NULL;
- DefElem *versionEl = NULL;
char *collcollate = NULL;
char *collctype = NULL;
char *collproviderstr = NULL;
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -96,8 +94,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
defelp = &providerEl;
else if (strcmp(defel->defname, "deterministic") == 0)
defelp = &deterministicEl;
- else if (strcmp(defel->defname, "version") == 0)
- defelp = &versionEl;
else
{
ereport(ERROR,
@@ -166,9 +162,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +208,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +216,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +266,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +523,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +583,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +644,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index d8cf87e6d0..5cbbe3ba2e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5232,9 +5222,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 627b026b19..73a9507b80 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3284,9 +3276,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4ff35095b8..c43b16037e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -838,7 +838,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10137,21 +10136,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 97cbaa3072..7453802fce 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1795,10 +1795,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2946,10 +2942,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3562,10 +3554,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 11d05c73ac..2c5f5bfa37 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1522,8 +1522,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1625,41 +1623,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index c2627bb630..d3b06094dd 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13662,12 +13662,10 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
- "collprovider, "
- "collversion, ");
+ "collprovider, ");
else
appendPQExpBufferStr(query,
- "'c' AS collprovider, "
- "NULL AS collversion, ");
+ "'c' AS collprovider, ");
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(query,
@@ -13728,24 +13726,6 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
appendStringLiteralAH(q, collctype, fout);
}
- /*
- * For binary upgrade, carry over the collation version. For normal
- * dump/restore, omit the version, so that it is computed upon restore.
- */
- if (dopt->binary_upgrade)
- {
- int i_collversion;
-
- i_collversion = PQfnumber(res, "collversion");
- if (!PQgetisnull(res, 0, i_collversion))
- {
- appendPQExpBufferStr(q, ", version = ");
- appendStringLiteralAH(q,
- PQgetvalue(res, 0, i_collversion),
- fout);
- }
- }
-
appendPQExpBufferStr(q, ");\n");
if (dopt->binary_upgrade)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 5e1ffafb91..7a23fb7529 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1873,17 +1873,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v26-0002-Add-pg_depend.refobjversion.patchtext/x-patch; charset=US-ASCII; name=v26-0002-Add-pg_depend.refobjversion.patchDownload
From b65542695ebb76604ed082758109f83b9e70b22d Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v26 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and maybe more.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 +++++++
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 62 insertions(+), 32 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 04fcb5fd33..f879a8c46e 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3293,6 +3293,18 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </para></entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index b33a2f94af..070b27c692 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1598,7 +1598,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1685,7 +1687,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1705,7 +1709,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2677,7 +2683,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 21cfdcace9..5d45e706cb 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 786672b1b6..4b2a58cf18 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1572,55 +1572,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v26-0003-Track-collation-versions-for-indexes.patchtext/x-patch; charset=US-ASCII; name=v26-0003-Track-collation-versions-for-indexes.patchDownload
From 551845e3df9b4a14d024101c50f9bc06e1334cad Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v26 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3985326df6..d55a5d358a 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2303,7 +2303,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2313,7 +2313,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3637,7 +3637,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d3b06094dd..55cd053214 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7002,7 +7010,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7038,7 +7048,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7063,7 +7128,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7102,7 +7169,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7137,7 +7206,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7168,7 +7239,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7202,7 +7275,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7242,6 +7317,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7267,6 +7344,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16500,7 +16579,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16567,6 +16647,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16595,6 +16679,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18579,6 +18678,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d951b4a36f..3e355dc19e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10339,6 +10339,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v26-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/x-patch; charset=US-ASCII; name=v26-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchDownload
From 712a6b7c4f42b878828655181f1f3f57ca26063f Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v26 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
Allow privileged users to declare that the currently installed collation
version, for a specific collation, is binary compatible with the one
that was installed when the index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 47 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 137 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index a5e3b06ee4..03355164b3 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e85aeb5502..ef2b896b6f 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3156,7 +3156,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 42330692e7..501cce686b 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -556,6 +557,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3926,6 +3928,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4093,6 +4099,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4659,6 +4671,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17477,3 +17494,33 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This overrides an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 5cbbe3ba2e..ebf9118f4f 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c43b16037e..68934bfeb1 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2594,6 +2594,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index eb018854a5..ec0ef1feae 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -814,6 +815,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1709,7 +1724,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1759,6 +1775,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7a23fb7529..423f33b72c 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1847,7 +1847,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1863,6 +1864,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v26-0005-Doc-Add-Collation-Versions-section.patchtext/x-patch; charset=US-ASCII; name=v26-0005-Doc-Add-Collation-Versions-section.patchDownload
From f7cd0bce75202b06c4ba75282ba6948496fb4ae1 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v26 5/5] Doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 4b4563c5b9..12a82ff18c 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be issued
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Wed, Jul 08, 2020 at 06:12:51PM +1200, Thomas Munro wrote:
I still wish I had a better idea than this:
+/* + * Returns whether the given index access method depend on a stable collation + * order. + */ +static bool +index_depends_stable_coll_order(Oid amoid) +{ + return (amoid != HASH_AM_OID && + strcmp(get_am_name(amoid), "bloom") != 0); +}I'm doing some more testing and looking for weird cases... More soon.
Wouldn't the normal way to track that a new field in IndexAmRoutine?
What you have here is not extensible.
--
Michael
Re: Collation versioning
On 2020-07-08 08:26, Michael Paquier wrote:
On Wed, Jul 08, 2020 at 06:12:51PM +1200, Thomas Munro wrote:
I still wish I had a better idea than this:
+/* + * Returns whether the given index access method depend on a stable collation + * order. + */ +static bool +index_depends_stable_coll_order(Oid amoid) +{ + return (amoid != HASH_AM_OID && + strcmp(get_am_name(amoid), "bloom") != 0); +}I'm doing some more testing and looking for weird cases... More soon.
Wouldn't the normal way to track that a new field in IndexAmRoutine?
What you have here is not extensible.
Yeah, this should be decided and communicated by the index AM somehow.
Perhaps it would also make sense to let the index AM handle the
differences between deterministic and nondeterministic collations. I
don't know how the bloom AM works, though, to determine whether that
makes sense.
In order not to derail this patch set I think it would be okay for now
to just include all index AMs in dependency tracking and invent a
mechanism later that excludes hash and bloom in an extensible manner.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On Thu, Jul 9, 2020 at 10:00 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 2020-07-08 08:26, Michael Paquier wrote:
On Wed, Jul 08, 2020 at 06:12:51PM +1200, Thomas Munro wrote:
I still wish I had a better idea than this:
+/* + * Returns whether the given index access method depend on a stable collation + * order. + */ +static bool +index_depends_stable_coll_order(Oid amoid) +{ + return (amoid != HASH_AM_OID && + strcmp(get_am_name(amoid), "bloom") != 0); +}I'm doing some more testing and looking for weird cases... More soon.
Wouldn't the normal way to track that a new field in IndexAmRoutine?
What you have here is not extensible.Yeah, this should be decided and communicated by the index AM somehow.
Perhaps it would also make sense to let the index AM handle the
differences between deterministic and nondeterministic collations. I
don't know how the bloom AM works, though, to determine whether that
makes sense.In order not to derail this patch set I think it would be okay for now
to just include all index AMs in dependency tracking and invent a
mechanism later that excludes hash and bloom in an extensible manner.
FTR I'll be happy to take care of that.
Re: Collation versioning
On Thu, Jul 9, 2020 at 11:13 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Thu, Jul 9, 2020 at 10:00 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:In order not to derail this patch set I think it would be okay for now
to just include all index AMs in dependency tracking and invent a
mechanism later that excludes hash and bloom in an extensible manner.FTR I'll be happy to take care of that.
Ok, thanks! Let's go with that.
Re: Collation versioning
On Fri, Jul 10, 2020 at 12:15:44PM +1200, Thomas Munro wrote:
On Thu, Jul 9, 2020 at 11:13 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Thu, Jul 9, 2020 at 10:00 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:In order not to derail this patch set I think it would be okay for now
to just include all index AMs in dependency tracking and invent a
mechanism later that excludes hash and bloom in an extensible manner.FTR I'll be happy to take care of that.
Ok, thanks! Let's go with that.
Thanks!
Here's a rebased v27 that removes the current approach to ignore indexes that
don't rely on a stable ordering. I'll start a new thread on that matter once
the infrastructure pieces will be committed.
Attachments:
v27-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 7e5dd2410ef003d8f528d7a21e8496f299babed3 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v27 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 -------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 88 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 24 +----
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 7 insertions(+), 343 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 26fda20d19..a28ae89e13 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2353,17 +2353,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index f766c1bc67..8432e790f2 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,11 +25438,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index bee6f0dd3c..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..5ad8886e60 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -61,14 +61,12 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *lcctypeEl = NULL;
DefElem *providerEl = NULL;
DefElem *deterministicEl = NULL;
- DefElem *versionEl = NULL;
char *collcollate = NULL;
char *collctype = NULL;
char *collproviderstr = NULL;
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -96,8 +94,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
defelp = &providerEl;
else if (strcmp(defel->defname, "deterministic") == 0)
defelp = &deterministicEl;
- else if (strcmp(defel->defname, "version") == 0)
- defelp = &versionEl;
else
{
ereport(ERROR,
@@ -166,9 +162,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +208,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +216,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +266,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +523,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +583,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +644,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 89c409de66..2218f6e3db 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5232,9 +5222,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e3f33c40be..40eb879176 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3284,9 +3276,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dbb47d4982..e432049f8e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -838,7 +838,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10137,21 +10136,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9b0c376c8c..fb8a964cc4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1795,10 +1795,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2946,10 +2942,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3562,10 +3554,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 07299dbc09..514e0fa0af 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1513,8 +1513,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1616,41 +1614,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..af8d38488e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13548,12 +13548,10 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
- "collprovider, "
- "collversion, ");
+ "collprovider, ");
else
appendPQExpBufferStr(query,
- "'c' AS collprovider, "
- "NULL AS collversion, ");
+ "'c' AS collprovider, ");
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(query,
@@ -13614,24 +13612,6 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
appendStringLiteralAH(q, collctype, fout);
}
- /*
- * For binary upgrade, carry over the collation version. For normal
- * dump/restore, omit the version, so that it is computed upon restore.
- */
- if (dopt->binary_upgrade)
- {
- int i_collversion;
-
- i_collversion = PQfnumber(res, "collversion");
- if (!PQgetisnull(res, 0, i_collversion))
- {
- appendPQExpBufferStr(q, ", version = ");
- appendStringLiteralAH(q,
- PQgetvalue(res, 0, i_collversion),
- fout);
- }
- }
-
appendPQExpBufferStr(q, ");\n");
if (dopt->binary_upgrade)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 151bcdb7ef..a8da438d22 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1873,17 +1873,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v27-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From 8145acbb54edc36123ef58b074b5a44988a5328e Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v27 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and maybe more.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 +++++++
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 62 insertions(+), 32 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index a28ae89e13..2567863dd9 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3293,6 +3293,18 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </para></entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..1a927377e7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1600,7 +1600,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1687,7 +1689,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1707,7 +1711,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2679,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 70baf03178..d86f85d142 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 786672b1b6..4b2a58cf18 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1572,55 +1572,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v27-0003-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From 1ae93375e2355f58d4a4a158d2bf50a62c3f1f1d Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v27 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From e12dd12f25d45228663daa10e951c3e1870ed12b Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v27 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
Allow privileged users to declare that the currently installed collation
version, for a specific collation, is binary compatible with the one
that was installed when the index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 47 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 137 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index a5e3b06ee4..03355164b3 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5cd3af3053..78bac65a7e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3130,7 +3130,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index cd989c95e5..2f931e3ca4 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -556,6 +557,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3926,6 +3928,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4093,6 +4099,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4659,6 +4671,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17426,3 +17443,33 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This overrides an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2218f6e3db..db35add7bb 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index e432049f8e..3199f9e7e1 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2594,6 +2594,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index f41785f11c..71e982ffa0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -820,6 +821,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1715,7 +1730,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1765,6 +1781,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index a8da438d22..0370ad6361 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1847,7 +1847,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1863,6 +1864,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 0075e85072..1843d71a38 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v27-0005-Doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From b59bb754e6d0a91f1ba1c504d2cf62a7edf38c52 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v27 5/5] Doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 4b4563c5b9..12a82ff18c 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be issued
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Thu, Aug 13, 2020 at 9:52 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
Here's a rebased v27 that removes the current approach to ignore indexes that
don't rely on a stable ordering. I'll start a new thread on that matter once
the infrastructure pieces will be committed.
Thanks Julien. I'm planning to do a bit more testing and review, and
then hopefully commit this next week. If anyone else has objections
to this design, now would be a good time to speak up.
Re: Collation versioning
On Fri, Aug 14, 2020 at 02:21:58PM +1200, Thomas Munro wrote:
Thanks Julien. I'm planning to do a bit more testing and review, and
then hopefully commit this next week. If anyone else has objections
to this design, now would be a good time to speak up.
The design to use pg_depend for the version string and rely on an
unknown state for indexes whose collations are unknown has a clear
consensus, so nothing to say about that. It looks like this will
benefit from using multi-INSERTs with pg_depend, actually.
I have read through the patch, and there are a couple of portions that
could be improved and/or simplified.
/*
- * Adjust all dependency records to come from a different object of the same type
* Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}The comment on top of the routine is wrong, as it could apply to
something else than indexes. Anyway, I don't think there is much
value in adding this API as the only part where this counts is
relation swapping for reindex concurrently. It could also be possible
that this breaks some extension code by making those static to
pg_depend.c.
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
All the callers of changeDependenciesOf() set the new argument to
true, making the false path dead, even if it just implies that the
argument is null. I would suggest to keep the original function
signature. If somebody needs a version where they don't want to
preserve the version, it could just be added later.+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
s/depedencies/dependencies/+ /*
+ * XXX For deterministic transaction, se should only track the
version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
Some cleanup needed here? Wouldn't it be better to address the issues
with stable ordering first?+ /* recordDependencyOnSingleRelExpr get rid of duplicated
entries */
s/get/gets/, incorrect grammar.
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
Something to do here?+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
Strange diff format here.+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
All the new functions in index.c should have more documentation and
comments to explain what they do.+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
I think that you could just use an array of ObjectAddresses here, fill
in a set of ObjectAddress objects and just call
recordMultipleDependencies() for all of them? Just create a set using
new_object_addresses(), register them with add_exact_object_address(),
and then finish the job with record_object_address_dependencies().
--
MichaelRe: Collation versioning
Hi Michael,
On Fri, Aug 14, 2020 at 04:37:46PM +0900, Michael Paquier wrote:
On Fri, Aug 14, 2020 at 02:21:58PM +1200, Thomas Munro wrote:
Thanks Julien. I'm planning to do a bit more testing and review, and
then hopefully commit this next week. If anyone else has objections
to this design, now would be a good time to speak up.The design to use pg_depend for the version string and rely on an
unknown state for indexes whose collations are unknown has a clear
consensus, so nothing to say about that. It looks like this will
benefit from using multi-INSERTs with pg_depend, actually.I have read through the patch, and there are a couple of portions that
could be improved and/or simplified./* - * Adjust all dependency records to come from a different object of the same type * Swap all dependencies of and on the old index to the new one, and + * vice-versa, while preserving any referenced version for the original owners. + * Note that a call to CommandCounterIncrement() would cause duplicate entries + * in pg_depend, so this should not be done. + */ +void +swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId) +{ + changeDependenciesOf(classId, firstObjectId, secondObjectId, true); + changeDependenciesOn(classId, firstObjectId, secondObjectId); + + changeDependenciesOf(classId, secondObjectId, firstObjectId, true); + changeDependenciesOn(classId, secondObjectId, firstObjectId); +}The comment on top of the routine is wrong, as it could apply to
something else than indexes. Anyway, I don't think there is much
value in adding this API as the only part where this counts is
relation swapping for reindex concurrently. It could also be possible
that this breaks some extension code by making those static to
pg_depend.c.
It seemed cleaner but ok, fixed.
-long +static long changeDependenciesOf(Oid classId, Oid oldObjectId, - Oid newObjectId) + Oid newObjectId, bool preserve_version) All the callers of changeDependenciesOf() set the new argument to true, making the false path dead, even if it just implies that the argument is null. I would suggest to keep the original function signature. If somebody needs a version where they don't want to preserve the version, it could just be added later.
Fixed.
+ * We don't want to record redundant depedencies that are used + * to track versions to avoid redundant warnings in case of s/depedencies/dependencies/+ /* + * XXX For deterministic transaction, se should only track the version + * if the AM relies on a stable ordering. + */ + if (determ_colls) + { + /* XXX check if the AM relies on a stable ordering */ + recordDependencyOnCollations(&myself, determ_colls, true); Some cleanup needed here? Wouldn't it be better to address the issues with stable ordering first?
Didn't we just agreed 3 mails ago to *not* take care of that in this patch, and
add an extensible solution for that later? I kept the XXX comment to make it
extra clear that this will be addressed.
+ /* recordDependencyOnSingleRelExpr get rid of duplicated
entries */
s/get/gets/, incorrect grammar.
Fixed.
+ /* XXX should we warn about "disappearing" versions? */ + if (current_version) + { Something to do here?
I'm not sure. This comment is to remind that we won't warn that an index
might get broken if say gnu_get_libc_version() stop giving a version number at
some point. I don't think that this will happen, but just in case there's a
comment to keep it in mind.
+ /* + * We now support versioning for the underlying collation library on + * this system, or previous version is unknown. + */ + if (!version || (strcmp(version, "") == 0 && strcmp(current_version, + "") != 0)) Strange diff format here.
That's what pgindent has been doing for some time, ie. indent at the same level
of the opening parenthesis.
+static char * +index_check_collation_version(const ObjectAddress *otherObject, + const char *version, + void *userdata) All the new functions in index.c should have more documentation and comments to explain what they do.
Fixed.
+ foreach(lc, collations) + { + ObjectAddress referenced; + + ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc)); + + recordMultipleDependencies(myself, &referenced, 1, + DEPENDENCY_NORMAL, record_version); + } I think that you could just use an array of ObjectAddresses here, fill in a set of ObjectAddress objects and just call recordMultipleDependencies() for all of them? Just create a set using new_object_addresses(), register them with add_exact_object_address(), and then finish the job with record_object_address_dependencies().
Fixed.
Attachments:
v28-0005-Doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 24d699a73dc18050fd5ac913085bd0dfcd871236 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v28 5/5] Doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 4b4563c5b9..12a82ff18c 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be issued
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
v28-0001-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 7e5dd2410ef003d8f528d7a21e8496f299babed3 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v28 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 -------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 88 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 24 +----
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 7 insertions(+), 343 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 26fda20d19..a28ae89e13 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2353,17 +2353,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index f766c1bc67..8432e790f2 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,11 +25438,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index bee6f0dd3c..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..5ad8886e60 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -61,14 +61,12 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *lcctypeEl = NULL;
DefElem *providerEl = NULL;
DefElem *deterministicEl = NULL;
- DefElem *versionEl = NULL;
char *collcollate = NULL;
char *collctype = NULL;
char *collproviderstr = NULL;
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -96,8 +94,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
defelp = &providerEl;
else if (strcmp(defel->defname, "deterministic") == 0)
defelp = &deterministicEl;
- else if (strcmp(defel->defname, "version") == 0)
- defelp = &versionEl;
else
{
ereport(ERROR,
@@ -166,9 +162,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +208,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +216,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +266,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +523,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +583,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +644,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 89c409de66..2218f6e3db 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5232,9 +5222,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e3f33c40be..40eb879176 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3284,9 +3276,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dbb47d4982..e432049f8e 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -838,7 +838,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10137,21 +10136,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9b0c376c8c..fb8a964cc4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1795,10 +1795,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2946,10 +2942,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3562,10 +3554,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 07299dbc09..514e0fa0af 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1513,8 +1513,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1616,41 +1614,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..af8d38488e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13548,12 +13548,10 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
- "collprovider, "
- "collversion, ");
+ "collprovider, ");
else
appendPQExpBufferStr(query,
- "'c' AS collprovider, "
- "NULL AS collversion, ");
+ "'c' AS collprovider, ");
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(query,
@@ -13614,24 +13612,6 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
appendStringLiteralAH(q, collctype, fout);
}
- /*
- * For binary upgrade, carry over the collation version. For normal
- * dump/restore, omit the version, so that it is computed upon restore.
- */
- if (dopt->binary_upgrade)
- {
- int i_collversion;
-
- i_collversion = PQfnumber(res, "collversion");
- if (!PQgetisnull(res, 0, i_collversion))
- {
- appendPQExpBufferStr(q, ", version = ");
- appendStringLiteralAH(q,
- PQgetvalue(res, 0, i_collversion),
- fout);
- }
- }
-
appendPQExpBufferStr(q, ");\n");
if (dopt->binary_upgrade)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 151bcdb7ef..a8da438d22 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1873,17 +1873,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v28-0002-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From 8145acbb54edc36123ef58b074b5a44988a5328e Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v28 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and maybe more.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 +++++++
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 62 insertions(+), 32 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index a28ae89e13..2567863dd9 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3293,6 +3293,18 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </para></entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..1a927377e7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1600,7 +1600,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1687,7 +1689,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1707,7 +1711,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2679,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 70baf03178..d86f85d142 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -79,8 +81,6 @@ recordMultipleDependencies(const ObjectAddress *depender,
/* Don't open indexes unless we need to make an update */
indstate = NULL;
- memset(nulls, false, sizeof(nulls));
-
for (i = 0; i < nreferenced; i++, referenced++)
{
/*
@@ -94,6 +94,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the Dependency. Note we don't bother to check for
* duplicate dependencies; there's no harm in them.
*/
+ memset(nulls, false, sizeof(nulls));
+ memset(values, 0, sizeof(values));
+
values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
@@ -101,9 +104,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
values[Anum_pg_depend_deptype - 1] = CharGetDatum((char) behavior);
-
tup = heap_form_tuple(dependDesc->rd_att, values, nulls);
/* fetch index info only when we know we need it */
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 786672b1b6..4b2a58cf18 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1572,55 +1572,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v28-0003-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From a8474e668de3fe41281f6dc0ef015b202cf9ad0e Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v28 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From 5d9e0802e54a21f49e99190f57a5cad02ef0f06a Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v28 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
Allow privileged users to declare that the currently installed collation
version, for a specific collation, is binary compatible with the one
that was installed when the index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 47 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 137 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index a5e3b06ee4..03355164b3 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e5155785b6..5b09c1deb4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3151,7 +3151,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index cd989c95e5..2f931e3ca4 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -556,6 +557,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3926,6 +3928,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4093,6 +4099,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4659,6 +4671,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17426,3 +17443,33 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This overrides an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2218f6e3db..db35add7bb 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index e432049f8e..3199f9e7e1 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2594,6 +2594,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index f41785f11c..71e982ffa0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -820,6 +821,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1715,7 +1730,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1765,6 +1781,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index a8da438d22..0370ad6361 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1847,7 +1847,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1863,6 +1864,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 0075e85072..1843d71a38 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
Re: Collation versioning
On Fri, Aug 14, 2020 at 11:02:35AM +0200, Julien Rouhaud wrote:
On Fri, Aug 14, 2020 at 04:37:46PM +0900, Michael Paquier wrote:
+ /* + * XXX For deterministic transaction, se should only track the version + * if the AM relies on a stable ordering. + */ + if (determ_colls) + { + /* XXX check if the AM relies on a stable ordering */ + recordDependencyOnCollations(&myself, determ_colls, true); Some cleanup needed here? Wouldn't it be better to address the issues with stable ordering first?Didn't we just agreed 3 mails ago to *not* take care of that in this patch, and
add an extensible solution for that later? I kept the XXX comment to make it
extra clear that this will be addressed.
FWIW, I tend to prefer the approach where we put in place the
necessary infrastructure first, and then have a feature rely on what
we think is the most correct. This way, we avoid having any moment in
the code history where we have something that we know from the start
is not covered.
The patch set needs a rebase. There are conflicts coming at least
from pg_depend.c where I switched the code to use multi-INSERTs for
catalog insertions.
--
Michael
Re: Collation versioning
On Mon, Sep 07, 2020 at 12:17:41PM +0900, Michael Paquier wrote:
On Fri, Aug 14, 2020 at 11:02:35AM +0200, Julien Rouhaud wrote:
On Fri, Aug 14, 2020 at 04:37:46PM +0900, Michael Paquier wrote:
+ /* + * XXX For deterministic transaction, se should only track the version + * if the AM relies on a stable ordering. + */ + if (determ_colls) + { + /* XXX check if the AM relies on a stable ordering */ + recordDependencyOnCollations(&myself, determ_colls, true); Some cleanup needed here? Wouldn't it be better to address the issues with stable ordering first?Didn't we just agreed 3 mails ago to *not* take care of that in this patch, and
add an extensible solution for that later? I kept the XXX comment to make it
extra clear that this will be addressed.FWIW, I tend to prefer the approach where we put in place the
necessary infrastructure first, and then have a feature rely on what
we think is the most correct. This way, we avoid having any moment in
the code history where we have something that we know from the start
is not covered.
I usually agree with that approach, I'm just afraid that getting a consensus on
the best way to do that will induce a lot of discussions, while this is
probably a corner case due to general usage of hash and bloom indexes.
Anyway, in order to make progress on that topic I attach an additional POC
commit to add the required infrastructure to handle this case in
v29-0001-Add-a-new-amnostablecollorder-flag-in-IndexAmRou.patch.
Here's the rationale for this new flag, extracted from the patch's commit
message:
Add a new amnostablecollorder flag in IndexAmRoutine.
This flag indicates if the access method does not rely on a stable collation
ordering for deterministic collation, i.e. would not be corrupted if the
underlying collation library changes its ordering. This is done this way to
make sure that if any external access method isn't updated to correctly setup
this flag it will be assumed to rely on a stable collation ordering as this
should be the case for the majority of access methods.
This flag will be useful for an upcoming commit that will add detection of
possibly corrupted index due to changed collation library version.
The patch set needs a rebase. There are conflicts coming at least
from pg_depend.c where I switched the code to use multi-INSERTs for
catalog insertions.
Fixed.
Attachments:
v29-0001-Add-a-new-amnostablecollorder-flag-in-IndexAmRou.patchtext/plain; charset=us-asciiDownload
From 00e7e77fe6bfe781890c5a97d202441048f3029d Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Tue, 8 Sep 2020 16:30:31 +0200
Subject: [PATCH v29 1/6] Add a new amnostablecollorder flag in IndexAmRoutine.
This flag indicates if the access method does not rely on a stable collation
ordering for deterministic collation, i.e. would not be corrupted if the
underlying collation library changes its ordering. This is done this way to
make sure that if any external access method isn't updated to correctly setup
this flag it will be assumed to rely on a stable collation ordering as this
should be the case for the majority of access methods.
This flag will be useful for an upcoming commit that will add detection of
possibly corrupted index due to changed collation library version.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
contrib/bloom/blutils.c | 1 +
doc/src/sgml/indexam.sgml | 2 ++
src/backend/access/brin/brin.c | 1 +
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gist.c | 1 +
src/backend/access/hash/hash.c | 1 +
src/backend/access/nbtree/nbtree.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/include/access/amapi.h | 2 ++
src/test/modules/dummy_index_am/dummy_index_am.c | 1 +
10 files changed, 12 insertions(+)
diff --git a/contrib/bloom/blutils.c b/contrib/bloom/blutils.c
index 26b9927c3a..9c9d0b994a 100644
--- a/contrib/bloom/blutils.c
+++ b/contrib/bloom/blutils.c
@@ -124,6 +124,7 @@ blhandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = true;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL | VACUUM_OPTION_PARALLEL_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index 390c49eb6a..ed5edc0aca 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -126,6 +126,8 @@ typedef struct IndexAmRoutine
bool amcaninclude;
/* does AM use maintenance_work_mem? */
bool amusemaintenanceworkmem;
+ /* does AM not rely on a stable ordering of deterministic collations? */
+ bool amnostablecollorder;
/* OR of parallel vacuum flags */
uint8 amparallelvacuumoptions;
/* type of data stored in index, or InvalidOid if variable */
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 1f72562c60..e687844829 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -105,6 +105,7 @@ brinhandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ef9b56fd36..de122ce805 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -56,6 +56,7 @@ ginhandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = true;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL | VACUUM_OPTION_PARALLEL_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 25b42e38f2..32350d8823 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -77,6 +77,7 @@ gisthandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = true;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL | VACUUM_OPTION_PARALLEL_COND_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 7c9ccf446c..84b3e8d557 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -74,6 +74,7 @@ hashhandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = true;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL;
amroutine->amkeytype = INT4OID;
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index c822b49a71..92da6ae5a4 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -126,6 +126,7 @@ bthandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = true;
amroutine->amcaninclude = true;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL | VACUUM_OPTION_PARALLEL_COND_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 64d3ba8288..56429556bd 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -59,6 +59,7 @@ spghandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL | VACUUM_OPTION_PARALLEL_COND_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 85b4766016..3f00033c01 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -243,6 +243,8 @@ typedef struct IndexAmRoutine
bool amcaninclude;
/* does AM use maintenance_work_mem? */
bool amusemaintenanceworkmem;
+ /* does AM not rely on a stable ordering of deterministic collations? */
+ bool amnostablecollorder;
/* OR of parallel vacuum flags. See vacuum.h for flags. */
uint8 amparallelvacuumoptions;
/* type of data stored in index, or InvalidOid if variable */
diff --git a/src/test/modules/dummy_index_am/dummy_index_am.c b/src/test/modules/dummy_index_am/dummy_index_am.c
index e97a32d5be..36c3696df2 100644
--- a/src/test/modules/dummy_index_am/dummy_index_am.c
+++ b/src/test/modules/dummy_index_am/dummy_index_am.c
@@ -297,6 +297,7 @@ dihandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions = VACUUM_OPTION_NO_PARALLEL;
amroutine->amkeytype = InvalidOid;
--
2.20.1
v29-0002-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 8ff111696473792f0aeaf64112999ec34888ee2f Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v29 2/6] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 -------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 88 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 24 +----
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 7 insertions(+), 343 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 508bea3bc6..1714d7116a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2361,17 +2361,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e2e618791e..ea2be6d819 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25483,11 +25483,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index bee6f0dd3c..c985b0de56 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..5ad8886e60 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -61,14 +61,12 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *lcctypeEl = NULL;
DefElem *providerEl = NULL;
DefElem *deterministicEl = NULL;
- DefElem *versionEl = NULL;
char *collcollate = NULL;
char *collctype = NULL;
char *collproviderstr = NULL;
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -96,8 +94,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
defelp = &providerEl;
else if (strcmp(defel->defname, "deterministic") == 0)
defelp = &deterministicEl;
- else if (strcmp(defel->defname, "version") == 0)
- defelp = &versionEl;
else
{
ereport(ERROR,
@@ -166,9 +162,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +208,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +216,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +266,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +523,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +583,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +644,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82..1fe4ceb7ad 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5231,9 +5221,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..0cf90ef33c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3283,9 +3275,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c5154b818c..c507719fa6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -838,7 +838,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10141,21 +10140,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9713a7ac41..1e979af18e 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1831,10 +1831,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2982,10 +2978,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3598,10 +3590,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 07299dbc09..514e0fa0af 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1513,8 +1513,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1616,41 +1614,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 784bceaec3..77139355e0 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13589,12 +13589,10 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
- "collprovider, "
- "collversion, ");
+ "collprovider, ");
else
appendPQExpBufferStr(query,
- "'c' AS collprovider, "
- "NULL AS collversion, ");
+ "'c' AS collprovider, ");
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(query,
@@ -13655,24 +13653,6 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
appendStringLiteralAH(q, collctype, fout);
}
- /*
- * For binary upgrade, carry over the collation version. For normal
- * dump/restore, omit the version, so that it is computed upon restore.
- */
- if (dopt->binary_upgrade)
- {
- int i_collversion;
-
- i_collversion = PQfnumber(res, "collversion");
- if (!PQgetisnull(res, 0, i_collversion))
- {
- appendPQExpBufferStr(q, ", version = ");
- appendStringLiteralAH(q,
- PQgetvalue(res, 0, i_collversion),
- fout);
- }
- }
-
appendPQExpBufferStr(q, ");\n");
if (dopt->binary_upgrade)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e83329fd6d..85e39b1b9d 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1874,17 +1874,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v29-0003-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From bf07d68c1543ae8f0ff898e844450429a6af56e2 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v29 3/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and maybe more.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 +++++++
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 62 insertions(+), 32 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 1714d7116a..e5489197f8 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3302,6 +3302,18 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </para></entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..1a927377e7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1600,7 +1600,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1687,7 +1689,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1707,7 +1711,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2679,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 454e569fa9..09c30b13e8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -115,6 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the dependency. Note we don't bother to check for duplicate
* dependencies; there's no harm in them.
*/
+ memset(slot[slot_stored_count]->tts_isnull, false,
+ slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+
slot[slot_stored_count]->tts_values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
@@ -122,9 +127,10 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot[slot_stored_count]->tts_values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
-
- memset(slot[slot_stored_count]->tts_isnull, false,
- slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+ if (version)
+ slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ slot[slot_stored_count]->tts_isnull[Anum_pg_depend_refobjversion - 1] = true;
ExecStoreVirtualTuple(slot[slot_stored_count]);
slot_stored_count++;
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 37e0d7ceab..9faad3d4ff 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1561,55 +1561,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v29-0004-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From 6f7561eaf3746467fb1a0bab3faf2aedf4592b04 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v29 4/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 264 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 208 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 +-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1655 insertions(+), 148 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index ea2be6d819..bcddca822c 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25483,7 +25483,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index b59c5697a3..527c847e27 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index 33af4ae02a..cbac0284c7 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..fc5140daa6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2336,7 +2336,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2346,7 +2346,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3665,7 +3665,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 117e3fdef7..a30e83ca39 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/rel.h"
@@ -1020,6 +1022,10 @@ index_create(Relation heapRelation,
ObjectAddress myself,
referenced;
ObjectAddresses *addrs;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1106,20 +1112,80 @@ index_create(Relation heapRelation,
/* placeholder for normal dependencies */
addrs = new_object_addresses();
- /* Store dependency on collations */
-
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- add_exact_object_address(&referenced, addrs);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic collation, we always track the dependency on the
+ * collation but only track the version if the AM relies on a stable
+ * ordering.
+ */
+ if (determ_colls)
+ {
+ IndexAmRoutine *routine;
+ bool track_version;
+
+ routine = GetIndexAmRoutineByAmId(accessMethodObjectId, false);
+ track_version = !routine->amnostablecollorder;
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1133,21 +1199,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1226,6 +1303,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3000,6 +3179,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3646,6 +3887,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 0d70cb0c3c..93774c9d21 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -362,7 +362,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 09c30b13e8..2bd9401e6a 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -66,6 +77,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
max_slots,
slot_init_count,
slot_stored_count;
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -96,12 +108,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (isObjectPinned(referenced, dependDesc))
+ if (!ignore_systempin && isObjectPinned(referenced, dependDesc))
continue;
if (slot_init_count < max_slots)
@@ -493,7 +542,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -501,8 +551,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -526,14 +575,47 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
- Form_pg_depend depform;
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
+
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
- depform->objid = newObjectId;
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -636,6 +718,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 0b04dff773..f2af6212e5 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ddeec870d8..e1016609dc 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index f9d0d67aa7..3a20dcdee5 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 96ecad02dd..96f8c67831 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 77139355e0..90535936ae 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -286,6 +287,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -387,6 +390,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -714,6 +718,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7034,7 +7042,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7070,7 +7080,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7095,7 +7160,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7134,7 +7201,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7169,7 +7238,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7200,7 +7271,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7234,7 +7307,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7274,6 +7349,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7299,6 +7376,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16360,7 +16439,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16427,6 +16507,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16455,6 +16539,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18441,6 +18540,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..623814d1c5 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if a partition, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 687509ba92..84466ab577 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10367,6 +10367,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1488bffa2b..8c54e12268 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 64c0c66859..d883ee58d1 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,14 +2065,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2092,14 +2094,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v29-0005-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From 9a8fa62328c0dc1a0bf0f7b67a673ae552f53826 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v29 5/6] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
Allow privileged users to declare that the currently installed collation
version, for a specific collation, is binary compatible with the one
that was installed when the index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 47 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 137 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index a5e3b06ee4..03355164b3 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index a30e83ca39..a549383bcd 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3179,7 +3179,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3e57c7f9e1..44060cdbe4 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -558,6 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3982,6 +3984,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4156,6 +4162,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4731,6 +4743,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17540,3 +17557,33 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This overrides an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1fe4ceb7ad..62727bb543 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c507719fa6..5d54e80827 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2594,6 +2594,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index f41785f11c..71e982ffa0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -820,6 +821,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1715,7 +1730,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1765,6 +1781,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 85e39b1b9d..45886f0e7a 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1848,7 +1848,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1864,6 +1865,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index db386c1b09..adc1dddda7 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2054,6 +2054,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index e93530af55..b3a75b29e6 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -823,6 +823,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v29-0006-Doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 541156e26101d21aab9cd2ce3bd34e1b0a148d78 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v29 6/6] Doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 4b4563c5b9..12a82ff18c 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be issued
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
Re: Julien Rouhaud
Here's the rationale for this new flag, extracted from the patch's commit
message:Add a new amnostablecollorder flag in IndexAmRoutine.
This flag indicates if the access method does not rely on a stable collation
ordering for deterministic collation, i.e. would not be corrupted if the
underlying collation library changes its ordering. This is done this way to
make sure that if any external access method isn't updated to correctly setup
this flag it will be assumed to rely on a stable collation ordering as this
should be the case for the majority of access methods.This flag will be useful for an upcoming commit that will add detection of
possibly corrupted index due to changed collation library version.
Hmm. Does that flag gain us much? What about non-collation locale
changes that might still affect indexes like lower() and the citext
extension? That still depends on locale changes, but that flag
wouldn't be able to help with "this index is (not) affected by this
locale change".
IOW, I think we should aim at simply tracking the version, and leave
it to the admin (with the help of supplied SQL queries) to either
rebuild indexes or waive them.
Or maybe I misunderstood the problem.
Christoph
Re: Collation versioning
On Tue, Sep 08, 2020 at 09:33:26PM +0200, Christoph Berg wrote:
Re: Julien Rouhaud
Here's the rationale for this new flag, extracted from the patch's commit
message:Add a new amnostablecollorder flag in IndexAmRoutine.
This flag indicates if the access method does not rely on a stable collation
ordering for deterministic collation, i.e. would not be corrupted if the
underlying collation library changes its ordering. This is done this way to
make sure that if any external access method isn't updated to correctly setup
this flag it will be assumed to rely on a stable collation ordering as this
should be the case for the majority of access methods.This flag will be useful for an upcoming commit that will add detection of
possibly corrupted index due to changed collation library version.Hmm. Does that flag gain us much? What about non-collation locale
changes that might still affect indexes like lower() and the citext
extension? That still depends on locale changes, but that flag
wouldn't be able to help with "this index is (not) affected by this
locale change".IOW, I think we should aim at simply tracking the version, and leave
it to the admin (with the help of supplied SQL queries) to either
rebuild indexes or waive them.Or maybe I misunderstood the problem.
I see your point, and indeed this isn't really clear how the flag will be used
given this description.
I guess that my idea is that how exactly an index depends on a stable
collation ordering isn't part of this flag definition, as it should be the same
for any access method.
In the commit that uses the infrastructure, the lack of stable ordering
requirement is only used for index key column, but not for any
expression, whether on index key or qual, because as you mention there's no
guarantee that the expression itself depends on a stable ordering or not.
There could be some improvements done for some simple case (like maybe md5() is
used often enough in index keys that it could be worth to detect), but nothing
in done to attempt that for now.
Re: Collation versioning
On 2020-09-08 21:33, Christoph Berg wrote:
IOW, I think we should aim at simply tracking the version, and leave
it to the admin (with the help of supplied SQL queries) to either
rebuild indexes or waive them.
It's certainly safer to track dependency for all indexes and then
carefully create exceptions afterwards.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On Thu, Sep 10, 2020 at 6:52 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 2020-09-08 21:33, Christoph Berg wrote:
IOW, I think we should aim at simply tracking the version, and leave
it to the admin (with the help of supplied SQL queries) to either
rebuild indexes or waive them.It's certainly safer to track dependency for all indexes and then
carefully create exceptions afterwards.
Do you mean storing the collation version even when those are not
relevant, and let client code (or backend command) deal with it? This
would require to store a dependency per index and column (or at least
if it's a column or an expression to avoid bloating the dependencies
too much), as it's otherwise impossible to know if a version mismatch
can be safely ignored or not.
I'm also wondering how much more complexity it would add to people who
want to actively monitor such mismatch using SQL queries.
Re: Collation versioning
On 2020-09-08 16:45, Julien Rouhaud wrote:
I usually agree with that approach, I'm just afraid that getting a consensus on
the best way to do that will induce a lot of discussions, while this is
probably a corner case due to general usage of hash and bloom indexes.Anyway, in order to make progress on that topic I attach an additional POC
commit to add the required infrastructure to handle this case in
v29-0001-Add-a-new-amnostablecollorder-flag-in-IndexAmRou.patch.
I'm confused now. I think we had mostly agreed on the v28 patch set,
without this additional AM flag. There was still some discussion on
what the AM flag's precise semantics should be. Do we want to work that
out first?
Btw., I'm uneasy about the term "stable collation order". "Stable" has
an established meaning for sorting. It's really about whether the AM
uses collations at all, right?
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Collation versioning
On Tue, Sep 15, 2020 at 2:26 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 2020-09-08 16:45, Julien Rouhaud wrote:
I usually agree with that approach, I'm just afraid that getting a consensus on
the best way to do that will induce a lot of discussions, while this is
probably a corner case due to general usage of hash and bloom indexes.Anyway, in order to make progress on that topic I attach an additional POC
commit to add the required infrastructure to handle this case in
v29-0001-Add-a-new-amnostablecollorder-flag-in-IndexAmRou.patch.I'm confused now. I think we had mostly agreed on the v28 patch set,
without this additional AM flag. There was still some discussion on
what the AM flag's precise semantics should be. Do we want to work that
out first?
That was my understanding too, but since Michael raised a concern I
wrote some initial implementation for that part. I'm assuming that
this new flag will raise some new discussion, and I hope this can be
discussed later, or at least in parallel, without interfering with the
rest of the patchset.
Btw., I'm uneasy about the term "stable collation order". "Stable" has
an established meaning for sorting. It's really about whether the AM
uses collations at all, right?
Well, at the AM level I guess it's only about whether it's using some
kind of sorting or not, as the collation information is really at the
opclass level. It makes me realize that this approach won't be able
to cope with an index built using (varchar|text)_pattern_ops, and
that's probably something we should handle correctly.
Re: Collation versioning
On Thu, Sep 17, 2020 at 5:41 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Tue, Sep 15, 2020 at 2:26 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:I'm confused now. I think we had mostly agreed on the v28 patch set,
without this additional AM flag. There was still some discussion on
what the AM flag's precise semantics should be. Do we want to work that
out first?That was my understanding too, but since Michael raised a concern I
wrote some initial implementation for that part. I'm assuming that
this new flag will raise some new discussion, and I hope this can be
discussed later, or at least in parallel, without interfering with the
rest of the patchset.
If we always record dependencies we'll have the option to invent
clever ways to ignore them during version checking in later releases.
Btw., I'm uneasy about the term "stable collation order". "Stable" has
an established meaning for sorting. It's really about whether the AM
uses collations at all, right?Well, at the AM level I guess it's only about whether it's using some
kind of sorting or not, as the collation information is really at the
opclass level. It makes me realize that this approach won't be able
to cope with an index built using (varchar|text)_pattern_ops, and
that's probably something we should handle correctly.
Hmm.
Re: Collation versioning
On Sun, Sep 20, 2020 at 6:36 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Thu, Sep 17, 2020 at 5:41 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Tue, Sep 15, 2020 at 2:26 PM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:I'm confused now. I think we had mostly agreed on the v28 patch set,
without this additional AM flag. There was still some discussion on
what the AM flag's precise semantics should be. Do we want to work that
out first?That was my understanding too, but since Michael raised a concern I
wrote some initial implementation for that part. I'm assuming that
this new flag will raise some new discussion, and I hope this can be
discussed later, or at least in parallel, without interfering with the
rest of the patchset.If we always record dependencies we'll have the option to invent
clever ways to ignore them during version checking in later releases.
But in any case we need to record the dependencies for all collations
right? The only difference is that we shouldn't record the collation
version if there's no risk of corruption if the underlying sort order
changes.
So while I want to address this part in pg14, if that wasn't the case
the problem would anyway be automatically fixed in the later version
by doing a reindex I think, as the version would be cleared.
There could still be a possible false positive warning in that case if
the lib is updated, but users could clear it with the infrastructure
proposed. Or alternatively if we add a new backend filter, say
REINDEX (COLLATION NOT CURRENT), we could add there additional
knowledge to ignore such cases.
Btw., I'm uneasy about the term "stable collation order". "Stable" has
an established meaning for sorting. It's really about whether the AM
uses collations at all, right?Well, at the AM level I guess it's only about whether it's using some
kind of sorting or not, as the collation information is really at the
opclass level. It makes me realize that this approach won't be able
to cope with an index built using (varchar|text)_pattern_ops, and
that's probably something we should handle correctly.Hmm.
On the other hand the *_pattern_ops are entirely hardcoded, and I
don't think that we'll ever have an extensible way to have this kind
of magic exception. So maybe having a flag at the am level is
acceptable?
Re: Collation versioning
On Sun, Sep 20, 2020 at 10:24:26AM +0800, Julien Rouhaud wrote:
On the other hand the *_pattern_ops are entirely hardcoded, and I
don't think that we'll ever have an extensible way to have this kind
of magic exception. So maybe having a flag at the am level is
acceptable?
Hearing no complaint, I kept the flag at the AM level and added hardcoded
exceptions for the *_pattern_ops opclasses to avoid false positive as much as
possible, and no false negative (at least that I'm aware of). I added many
indexes to the regression tests to make sure that all the cases are correctly
handled.
Unfortunately, there's still one case that can't be fixed easily. Here's an
example of such case:
CREATE INDEX ON sometable ((collatable_col || collatable_col) text_pattern_ops)
In this case when iterating over the key columns, the current patch notices
that a dependency on the collation should be added but that the version
shouldn't be tracked, as it's text_pattern_ops. But then the expression itself
is processed, and it also see that a dependency on the collation should be
addedd. However, it has no way to know that it should not update the
previously recorded dependency to start tracking the version, as the expression
could really depends on a stable order.
So we end up with a single dependency (which is what we want I think), but
which will report false positive warning in case of collation lib update.
v30 attached.
Attachments:
v30-0001-Add-a-new-amnostablecollorder-flag-in-IndexAmRou.patchtext/plain; charset=us-asciiDownload
From a85a13277049613ce5258e1a11419e3c69e98ccd Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Tue, 8 Sep 2020 16:30:31 +0200
Subject: [PATCH v30 1/6] Add a new amnostablecollorder flag in IndexAmRoutine.
This flag indicates if the access method does not rely on a stable collation
ordering for deterministic collation, i.e. would not be corrupted if the
underlying collation library changes its ordering. This is done this way to
make sure that if any external access method isn't updated to correctly setup
this flag it will be assumed to rely on a stable collation ordering as this
should be the case for the majority of access methods.
This flag will be useful for an upcoming commit that will add detection of
possibly corrupted index due to changed collation library version.
Author: Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
contrib/bloom/blutils.c | 1 +
doc/src/sgml/indexam.sgml | 2 ++
src/backend/access/brin/brin.c | 1 +
src/backend/access/gin/ginutil.c | 1 +
src/backend/access/gist/gist.c | 1 +
src/backend/access/hash/hash.c | 1 +
src/backend/access/nbtree/nbtree.c | 1 +
src/backend/access/spgist/spgutils.c | 1 +
src/include/access/amapi.h | 2 ++
src/test/modules/dummy_index_am/dummy_index_am.c | 1 +
10 files changed, 12 insertions(+)
diff --git a/contrib/bloom/blutils.c b/contrib/bloom/blutils.c
index 26b9927c3a..9c9d0b994a 100644
--- a/contrib/bloom/blutils.c
+++ b/contrib/bloom/blutils.c
@@ -124,6 +124,7 @@ blhandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = true;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL | VACUUM_OPTION_PARALLEL_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index 390c49eb6a..ed5edc0aca 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -126,6 +126,8 @@ typedef struct IndexAmRoutine
bool amcaninclude;
/* does AM use maintenance_work_mem? */
bool amusemaintenanceworkmem;
+ /* does AM not rely on a stable ordering of deterministic collations? */
+ bool amnostablecollorder;
/* OR of parallel vacuum flags */
uint8 amparallelvacuumoptions;
/* type of data stored in index, or InvalidOid if variable */
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 1f72562c60..e687844829 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -105,6 +105,7 @@ brinhandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/src/backend/access/gin/ginutil.c b/src/backend/access/gin/ginutil.c
index ef9b56fd36..de122ce805 100644
--- a/src/backend/access/gin/ginutil.c
+++ b/src/backend/access/gin/ginutil.c
@@ -56,6 +56,7 @@ ginhandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = true;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL | VACUUM_OPTION_PARALLEL_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/src/backend/access/gist/gist.c b/src/backend/access/gist/gist.c
index 25b42e38f2..32350d8823 100644
--- a/src/backend/access/gist/gist.c
+++ b/src/backend/access/gist/gist.c
@@ -77,6 +77,7 @@ gisthandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = true;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL | VACUUM_OPTION_PARALLEL_COND_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 7c9ccf446c..84b3e8d557 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -74,6 +74,7 @@ hashhandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = true;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL;
amroutine->amkeytype = INT4OID;
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index c822b49a71..92da6ae5a4 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -126,6 +126,7 @@ bthandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = true;
amroutine->amcaninclude = true;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL | VACUUM_OPTION_PARALLEL_COND_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/src/backend/access/spgist/spgutils.c b/src/backend/access/spgist/spgutils.c
index 64d3ba8288..56429556bd 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -59,6 +59,7 @@ spghandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions =
VACUUM_OPTION_PARALLEL_BULKDEL | VACUUM_OPTION_PARALLEL_COND_CLEANUP;
amroutine->amkeytype = InvalidOid;
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 85b4766016..3f00033c01 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -243,6 +243,8 @@ typedef struct IndexAmRoutine
bool amcaninclude;
/* does AM use maintenance_work_mem? */
bool amusemaintenanceworkmem;
+ /* does AM not rely on a stable ordering of deterministic collations? */
+ bool amnostablecollorder;
/* OR of parallel vacuum flags. See vacuum.h for flags. */
uint8 amparallelvacuumoptions;
/* type of data stored in index, or InvalidOid if variable */
diff --git a/src/test/modules/dummy_index_am/dummy_index_am.c b/src/test/modules/dummy_index_am/dummy_index_am.c
index e97a32d5be..36c3696df2 100644
--- a/src/test/modules/dummy_index_am/dummy_index_am.c
+++ b/src/test/modules/dummy_index_am/dummy_index_am.c
@@ -297,6 +297,7 @@ dihandler(PG_FUNCTION_ARGS)
amroutine->amcanparallel = false;
amroutine->amcaninclude = false;
amroutine->amusemaintenanceworkmem = false;
+ amroutine->amnostablecollorder = false;
amroutine->amparallelvacuumoptions = VACUUM_OPTION_NO_PARALLEL;
amroutine->amkeytype = InvalidOid;
--
2.20.1
v30-0002-Remove-pg_collation.collversion.patchtext/plain; charset=us-asciiDownload
From 1ed46082e0e9a096652a6d781e3467ae91530080 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v30 2/6] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. A later patch
will add version tracking for individual database objects that depend
on collations.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 -------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 88 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 24 +----
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 7 insertions(+), 343 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index de9bacd34f..7834e0eee0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2361,17 +2361,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 461b748d89..1f33f99040 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25462,11 +25462,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index af9ff2867b..65429aabe2 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..5ad8886e60 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -61,14 +61,12 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *lcctypeEl = NULL;
DefElem *providerEl = NULL;
DefElem *deterministicEl = NULL;
- DefElem *versionEl = NULL;
char *collcollate = NULL;
char *collctype = NULL;
char *collproviderstr = NULL;
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -96,8 +94,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
defelp = &providerEl;
else if (strcmp(defel->defname, "deterministic") == 0)
defelp = &deterministicEl;
- else if (strcmp(defel->defname, "version") == 0)
- defelp = &versionEl;
else
{
ereport(ERROR,
@@ -166,9 +162,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +208,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +216,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +266,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +523,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +583,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +644,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82..1fe4ceb7ad 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3226,16 +3226,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5231,9 +5221,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..0cf90ef33c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3283,9 +3275,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 17653ef3a7..38570a6271 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -835,7 +835,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10138,21 +10137,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..f398027fa6 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1842,10 +1842,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2993,10 +2989,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3609,10 +3601,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 07299dbc09..514e0fa0af 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1513,8 +1513,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1616,41 +1614,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f021bb72f4..2174d66c1f 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13582,12 +13582,10 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
- "collprovider, "
- "collversion, ");
+ "collprovider, ");
else
appendPQExpBufferStr(query,
- "'c' AS collprovider, "
- "NULL AS collversion, ");
+ "'c' AS collprovider, ");
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(query,
@@ -13648,24 +13646,6 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
appendStringLiteralAH(q, collctype, fout);
}
- /*
- * For binary upgrade, carry over the collation version. For normal
- * dump/restore, omit the version, so that it is computed upon restore.
- */
- if (dopt->binary_upgrade)
- {
- int i_collversion;
-
- i_collversion = PQfnumber(res, "collversion");
- if (!PQgetisnull(res, 0, i_collversion))
- {
- appendPQExpBufferStr(q, ", version = ");
- appendStringLiteralAH(q,
- PQgetvalue(res, 0, i_collversion),
- fout);
- }
- }
-
appendPQExpBufferStr(q, ");\n");
if (dopt->binary_upgrade)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..32dc7cd5ef 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1875,17 +1875,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v30-0003-Add-pg_depend.refobjversion.patchtext/plain; charset=us-asciiDownload
From 7efecfe11150515cb73761d4b35c7795ba4f8de4 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v30 3/6] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and maybe more.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 +++++++
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 62 insertions(+), 32 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 7834e0eee0..90161ef3df 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3302,6 +3302,18 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </para></entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..1a927377e7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1600,7 +1600,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1687,7 +1689,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1707,7 +1711,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2679,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 454e569fa9..09c30b13e8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -115,6 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the dependency. Note we don't bother to check for duplicate
* dependencies; there's no harm in them.
*/
+ memset(slot[slot_stored_count]->tts_isnull, false,
+ slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+
slot[slot_stored_count]->tts_values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
@@ -122,9 +127,10 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot[slot_stored_count]->tts_values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
-
- memset(slot[slot_stored_count]->tts_isnull, false,
- slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+ if (version)
+ slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ slot[slot_stored_count]->tts_isnull[Anum_pg_depend_refobjversion - 1] = true;
ExecStoreVirtualTuple(slot[slot_stored_count]);
slot_stored_count++;
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 118b282d1c..81550fc11d 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1559,55 +1559,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v30-0004-Track-collation-versions-for-indexes.patchtext/plain; charset=us-asciiDownload
From 5aeeec4586d40ad0625a558ce192c43894660799 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v30 4/6] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 +
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 219 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 307 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 262 ++++++++++++++-
src/backend/catalog/pg_type.c | 69 ++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 ++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 +-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 187 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 126 +++++++
37 files changed, 1803 insertions(+), 148 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 1f33f99040..86f2e16533 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25462,7 +25462,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index b59c5697a3..527c847e27 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index fa43e3a972..ef047a8945 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..c93fdbd7d5 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,37 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ eliminate_duplicate_dependencies(addrs);
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1698,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1716,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1744,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1811,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1833,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1856,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1895,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1963,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2058,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2147,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2179,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2193,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2207,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2297,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2443,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2467,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2865,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..fc5140daa6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2336,7 +2336,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2346,7 +2346,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3665,7 +3665,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..4c8a221e0d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/rel.h"
@@ -1020,6 +1022,11 @@ index_create(Relation heapRelation,
ObjectAddress myself,
referenced;
ObjectAddresses *addrs;
+ ListCell *lc;
+ List *colls = NIL,
+ *colls_pattern_ops = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1106,20 +1113,122 @@ index_create(Relation heapRelation,
/* placeholder for normal dependencies */
addrs = new_object_addresses();
- /* Store dependency on collations */
-
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- add_exact_object_address(&referenced, addrs);
+ Oid opclass = classObjectId[i];
+
+ /*
+ * The *_pattern_ops opclasses are special, as they're known
+ * to not rely on the collation sort order.
+ */
+ if (opclass == TEXT_BTREE_PATTERN_OPS_OID ||
+ opclass == VARCHAR_BTREE_PATTERN_OPS_OID ||
+ opclass == BPCHAR_BTREE_PATTERN_OPS_OID)
+ colls_pattern_ops = lappend_oid(colls_pattern_ops, colloid);
+ else
+ colls = lappend_oid(colls, colloid);
+ }
+ /*
+ * Required for e.g. custom types having multiple underlying
+ * collations. Note that none of the *_pattern_ops can be used in
+ * such constructs.
+ */
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /* Check if any collation exist for both a *_pattern_ops opclass and
+ * another one. In that case, we have to create a single dependency
+ * with version tracked.
+ */
+ if (colls_pattern_ops != NIL && colls != NIL)
+ colls_pattern_ops = list_difference_oid(colls_pattern_ops, colls);
+
+ /*
+ * Record the dependencies for collation declares with any of the
+ * *_pattern_ops opclass, without version tracking.
+ */
+ if (colls_pattern_ops != NIL)
+ {
+ recordDependencyOnCollations(&myself, colls_pattern_ops, false);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * Then split the dependencies for collations that were not declared
+ * with any of the *_pattern_ops opclass on whether they're
+ * deterministic or not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = lappend_oid(determ_colls, c);
+ else
+ nondeterm_colls = lappend_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic collation, we always track the dependency on the
+ * collation but only track the version if the AM relies on a stable
+ * ordering.
+ */
+ if (determ_colls != NIL)
+ {
+ IndexAmRoutine *routine;
+ bool track_version;
+
+ routine = GetIndexAmRoutineByAmId(accessMethodObjectId, false);
+ track_version = !routine->amnostablecollorder;
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls != NIL)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1133,21 +1242,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1226,6 +1346,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3000,6 +3222,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3635,6 +3919,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 0d70cb0c3c..93774c9d21 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -362,7 +362,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 09c30b13e8..89c866cb04 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,15 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ char *version);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +52,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -66,6 +78,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
max_slots,
slot_init_count,
slot_stored_count;
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -96,12 +109,55 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ * However, we may have already recorded a dependency on a
+ * collation that didn't required to track the version (if any
+ * of the *_pattern_ops opclasses was used). If we now see a
+ * dependency on the same collation that requires to track the
+ * version, we need to update it with the current version,
+ * which is done by dependencyExists().
+ */
+ version = get_collation_version_for_oid(referenced->objectId);
+ if (dependencyExists(depender, referenced, version))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (isObjectPinned(referenced, dependDesc))
+ if (!ignore_systempin && isObjectPinned(referenced, dependDesc))
continue;
if (slot_init_count < max_slots)
@@ -493,7 +549,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -501,8 +558,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -526,14 +582,47 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
- Form_pg_depend depform;
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -636,6 +725,151 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ *
+ * If a record is found, also update the stored dependency version if it wasn't
+ * previously tracked.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ char *version)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ char *cur_version = NULL;
+ Datum depversion;
+ bool isnull;
+
+ ret = true;
+
+ /*
+ * We only need the retrieve the current version if a new version
+ * was passed.
+ */
+ if (version)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+ }
+
+ /* Track version if it wasn't yet and we have a new version. */
+ if (version && !cur_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ else
+ {
+ Assert(strcmp(cur_version, version) == 0);
+ }
+
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 0b04dff773..f2af6212e5 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ddeec870d8..e1016609dc 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index f9d0d67aa7..3a20dcdee5 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9061af81a3..153adc157d 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5934,6 +5935,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2174d66c1f..cc5accc84c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7024,7 +7032,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7060,7 +7070,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7085,7 +7150,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7124,7 +7191,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7159,7 +7228,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7190,7 +7261,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7224,7 +7297,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7264,6 +7339,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7289,6 +7366,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16353,7 +16432,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16420,6 +16500,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16448,6 +16532,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18434,6 +18533,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..623814d1c5 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if a partition, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f48f5fb4d9..daf29aa6c5 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10367,6 +10367,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1488bffa2b..8c54e12268 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..68fe86cf89 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,193 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+-----------------------------------+------------+---------------------
+ icuidx00_val | "fr-x-icu" | up to date
+ icuidx00_val_val | "fr-x-icu" | up to date
+ icuidx00_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val | "fr-x-icu" | up to date
+ icuidx00_val_val_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_where | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(59 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..6cb7786e13 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,14 +2065,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2092,14 +2094,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..fbe220df41 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,132 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v30-0005-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/plain; charset=us-asciiDownload
From ef054165719aadf4ae4edf4aa6e5bea7226818f7 Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v30 5/6] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
Allow privileged users to declare that the currently installed collation
version, for a specific collation, is binary compatible with the one
that was installed when the index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 47 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 137 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index a5e3b06ee4..03355164b3 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4c8a221e0d..71776ddc5e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3222,7 +3222,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 16285ad09f..a9e2ded0b6 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -559,6 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3983,6 +3985,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4157,6 +4163,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4733,6 +4745,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17577,3 +17594,33 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This overrides an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1fe4ceb7ad..62727bb543 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3217,6 +3217,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 38570a6271..1c1b37a3f0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2591,6 +2591,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9c6f5ecb6a..b1bef416b8 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -820,6 +821,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1715,7 +1730,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1765,6 +1781,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 32dc7cd5ef..266519096e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1849,7 +1849,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1865,6 +1866,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 68fe86cf89..5d2c2e7e6f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2084,6 +2084,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index fbe220df41..c9afb88f7d 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -843,6 +843,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v30-0006-Doc-Add-Collation-Versions-section.patchtext/plain; charset=us-asciiDownload
From 2e2684c9673f81343bfe3cd5cfba6f4a98751897 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v30 6/6] Doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 2745b44417..9008d094bb 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be issued
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems).
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Thu, Sep 24, 2020 at 9:49 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Sun, Sep 20, 2020 at 10:24:26AM +0800, Julien Rouhaud wrote:
On the other hand the *_pattern_ops are entirely hardcoded, and I
don't think that we'll ever have an extensible way to have this kind
of magic exception. So maybe having a flag at the am level is
acceptable?Hearing no complaint, I kept the flag at the AM level and added hardcoded
exceptions for the *_pattern_ops opclasses to avoid false positive as much as
possible, and no false negative (at least that I'm aware of). I added many
indexes to the regression tests to make sure that all the cases are correctly
handled.Unfortunately, there's still one case that can't be fixed easily. Here's an
example of such case:CREATE INDEX ON sometable ((collatable_col || collatable_col) text_pattern_ops)
I think we should try to get the basic feature into the tree, and then
look at these kinds of subtleties as later improvements. Here's a new
version with the following changes:
1. Update the doc patch to mention that this stuff now works on
Windows too (see commit 352f6f2d).
2. Drop non_deterministic_only argument for from GetTypeCollations();
it was unused.
3. Drop that "stable collation order" field at the AM level for now.
This means that we'll warn you about collation versions changes for
hash and bloom indexes, even when it's technically unnecessary, for
now.
The pattern ops stuff seems straightforward however, so let's keep
that bit in the initial commit of the feature. That's a finite set of
hard coded op classes which exist specifically to ignore collations.
Attachments:
v31-0001-Remove-pg_collation.collversion.patchtext/x-patch; charset=US-ASCII; name=v31-0001-Remove-pg_collation.collversion.patchDownload
From b3f4770eefe93e0e540461d2d8ec35ee02c6613b Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:15:28 -0400
Subject: [PATCH v31 1/5] Remove pg_collation.collversion.
While this allowed the system to detect changes in collation versions,
it didn't help us track what had to be done about it. Later patches
will add a new mechanism.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 -------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 88 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 24 +----
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 7 insertions(+), 343 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 5bd54cb218..d56b3f8170 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2361,17 +2361,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c99499e52b..3b3d7d1c0e 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25481,11 +25481,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index af9ff2867b..65429aabe2 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..5ad8886e60 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -61,14 +61,12 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *lcctypeEl = NULL;
DefElem *providerEl = NULL;
DefElem *deterministicEl = NULL;
- DefElem *versionEl = NULL;
char *collcollate = NULL;
char *collctype = NULL;
char *collproviderstr = NULL;
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -96,8 +94,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
defelp = &providerEl;
else if (strcmp(defel->defname, "deterministic") == 0)
defelp = &deterministicEl;
- else if (strcmp(defel->defname, "version") == 0)
- defelp = &versionEl;
else
{
ereport(ERROR,
@@ -166,9 +162,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +208,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +216,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +266,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +523,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +583,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +644,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..ac8b57109c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3224,16 +3224,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5229,9 +5219,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..0cf90ef33c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3283,9 +3275,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..60cf7242a3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -835,7 +835,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10169,21 +10168,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..f398027fa6 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1842,10 +1842,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2993,10 +2989,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3609,10 +3601,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 07299dbc09..514e0fa0af 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1513,8 +1513,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1616,41 +1614,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ff45e3fb8c..1d09acb64c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13586,12 +13586,10 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
- "collprovider, "
- "collversion, ");
+ "collprovider, ");
else
appendPQExpBufferStr(query,
- "'c' AS collprovider, "
- "NULL AS collversion, ");
+ "'c' AS collprovider, ");
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(query,
@@ -13652,24 +13650,6 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
appendStringLiteralAH(q, collctype, fout);
}
- /*
- * For binary upgrade, carry over the collation version. For normal
- * dump/restore, omit the version, so that it is computed upon restore.
- */
- if (dopt->binary_upgrade)
- {
- int i_collversion;
-
- i_collversion = PQfnumber(res, "collversion");
- if (!PQgetisnull(res, 0, i_collversion))
- {
- appendPQExpBufferStr(q, ", version = ");
- appendStringLiteralAH(q,
- PQgetvalue(res, 0, i_collversion),
- fout);
- }
- }
-
appendPQExpBufferStr(q, ");\n");
if (dopt->binary_upgrade)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..32dc7cd5ef 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1875,17 +1875,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v31-0002-Add-pg_depend.refobjversion.patchtext/x-patch; charset=US-ASCII; name=v31-0002-Add-pg_depend.refobjversion.patchDownload
From 0e2ba5aa57760fa88aef75f6120e17c5f2302159 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:03 -0400
Subject: [PATCH v31 2/5] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. Later commits will be able to use this to record dependencies
on collation versions for indexes and maybe more.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 12 +++++++
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 62 insertions(+), 32 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index d56b3f8170..3b47441424 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3302,6 +3302,18 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object. The only current use of
+ <structfield>refobjversion</structfield> is to record dependencies
+ between indexes and collation versions.
+ </para></entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..1a927377e7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1600,7 +1600,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1687,7 +1689,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1707,7 +1711,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2679,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 454e569fa9..09c30b13e8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -115,6 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the dependency. Note we don't bother to check for duplicate
* dependencies; there's no harm in them.
*/
+ memset(slot[slot_stored_count]->tts_isnull, false,
+ slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+
slot[slot_stored_count]->tts_values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
@@ -122,9 +127,10 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot[slot_stored_count]->tts_values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
-
- memset(slot[slot_stored_count]->tts_isnull, false,
- slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+ if (version)
+ slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ slot[slot_stored_count]->tts_isnull[Anum_pg_depend_refobjversion - 1] = true;
ExecStoreVirtualTuple(slot[slot_stored_count]);
slot_stored_count++;
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index ee3bfa82f4..daf60d0c77 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1539,55 +1539,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v31-0003-Track-collation-versions-for-indexes.patchtext/x-patch; charset=US-ASCII; name=v31-0003-Track-collation-versions-for-indexes.patchDownload
From bc3ad8d8363e68f00ff38bf7b894611f6c9a0f15 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 28 May 2019 14:16:29 -0400
Subject: [PATCH v31 3/5] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 217 ++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 266 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 262 ++++++++++++++++-
src/backend/catalog/pg_type.c | 60 ++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 +-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 3 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 187 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 126 +++++++++
37 files changed, 1750 insertions(+), 147 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 3b3d7d1c0e..be90c92e5f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25481,7 +25481,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index b59c5697a3..527c847e27 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index fa43e3a972..ef047a8945 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..cab552eb32 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,37 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ eliminate_duplicate_dependencies(addrs);
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1698,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1716,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1744,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1811,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1833,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1856,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1895,47 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+
+ /*
+ * Otherwise, it may be a composite type having underlying
+ * collations.
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1961,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2056,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2145,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2177,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2191,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2205,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2295,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2441,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2465,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2863,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..fc5140daa6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2336,7 +2336,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2346,7 +2346,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3665,7 +3665,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..a014bfd7c6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/rel.h"
@@ -1020,6 +1022,8 @@ index_create(Relation heapRelation,
ObjectAddress myself,
referenced;
ObjectAddresses *addrs;
+ List *colls = NIL,
+ *colls_pattern_ops = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1106,20 +1110,84 @@ index_create(Relation heapRelation,
/* placeholder for normal dependencies */
addrs = new_object_addresses();
- /* Store dependency on collations */
-
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- add_exact_object_address(&referenced, addrs);
+ Oid opclass = classObjectId[i];
+
+ /*
+ * The *_pattern_ops opclasses are special, as they're known
+ * to not rely on the collation sort order.
+ */
+ if (opclass == TEXT_BTREE_PATTERN_OPS_OID ||
+ opclass == VARCHAR_BTREE_PATTERN_OPS_OID ||
+ opclass == BPCHAR_BTREE_PATTERN_OPS_OID)
+ colls_pattern_ops = lappend_oid(colls_pattern_ops, colloid);
+ else
+ colls = lappend_oid(colls, colloid);
+ }
+ /*
+ * Required for e.g. custom types having multiple underlying
+ * collations. Note that none of the *_pattern_ops can be used in
+ * such constructs.
+ */
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid));
}
}
+ /* Check if any collation exist for both a *_pattern_ops opclass and
+ * another one. In that case, we have to create a single dependency
+ * with version tracked.
+ */
+ if (colls_pattern_ops != NIL && colls != NIL)
+ colls_pattern_ops = list_difference_oid(colls_pattern_ops, colls);
+
+ /*
+ * Record the dependencies for collation declares with any of the
+ * *_pattern_ops opclass, without version tracking.
+ */
+ if (colls_pattern_ops != NIL)
+ {
+ recordDependencyOnCollations(&myself, colls_pattern_ops, false);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * Record the rest with collation versions.
+ *
+ * XXX With more infrastructure, we could suppress collation versions
+ * for some cases like hash indexes with deterministic collations,
+ * because they will never need to order strings.
+ */
+ if (colls != NIL)
+ {
+ recordDependencyOnCollations(&myself, colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1133,21 +1201,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1226,6 +1305,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3000,6 +3181,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3635,6 +3878,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 0d70cb0c3c..93774c9d21 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -362,7 +362,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 09c30b13e8..89c866cb04 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,15 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ char *version);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +52,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -66,6 +78,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
max_slots,
slot_init_count,
slot_stored_count;
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -96,12 +109,55 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ * However, we may have already recorded a dependency on a
+ * collation that didn't required to track the version (if any
+ * of the *_pattern_ops opclasses was used). If we now see a
+ * dependency on the same collation that requires to track the
+ * version, we need to update it with the current version,
+ * which is done by dependencyExists().
+ */
+ version = get_collation_version_for_oid(referenced->objectId);
+ if (dependencyExists(depender, referenced, version))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (isObjectPinned(referenced, dependDesc))
+ if (!ignore_systempin && isObjectPinned(referenced, dependDesc))
continue;
if (slot_init_count < max_slots)
@@ -493,7 +549,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -501,8 +558,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -526,14 +582,47 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
- Form_pg_depend depform;
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -636,6 +725,151 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ *
+ * If a record is found, also update the stored dependency version if it wasn't
+ * previously tracked.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ char *version)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ char *cur_version = NULL;
+ Datum depversion;
+ bool isnull;
+
+ ret = true;
+
+ /*
+ * We only need the retrieve the current version if a new version
+ * was passed.
+ */
+ if (version)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+ }
+
+ /* Track version if it wasn't yet and we have a new version. */
+ if (version && !cur_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ else
+ {
+ Assert(strcmp(cur_version, version) == 0);
+ }
+
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 0b04dff773..99b186484a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,65 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/*
+ * Get a list of all distinct collations oid that the given type depends on.
+ */
+List *
+GetTypeCollations(Oid typeoid)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result, GetTypeCollations(rangeid));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ddeec870d8..e1016609dc 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index f9d0d67aa7..3a20dcdee5 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9061af81a3..153adc157d 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5934,6 +5935,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..a1151e3b31 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -179,6 +179,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1d09acb64c..b82c16b151 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +388,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -712,6 +716,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7028,7 +7036,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7064,7 +7074,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7089,7 +7154,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7128,7 +7195,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7163,7 +7232,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7194,7 +7265,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7228,7 +7301,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7268,6 +7343,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7293,6 +7370,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16359,7 +16438,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16426,6 +16506,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16454,6 +16538,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18438,6 +18537,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..623814d1c5 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if a partition, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index bbcac69d48..7abfa4d65d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10387,6 +10387,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..f3061bca57 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index 9774f534d9..14cde4f5ba 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index ebcaeb44fe..c1b5d4aabd 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -769,10 +769,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..a7d7249cfa 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,193 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+-----------------------------------+------------+---------------------
+ icuidx00_val | "fr-x-icu" | up to date
+ icuidx00_val_val | "fr-x-icu" | up to date
+ icuidx00_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val | "fr-x-icu" | up to date
+ icuidx00_val_val_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_where | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(59 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..6cb7786e13 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,14 +2065,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2092,14 +2094,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..fbe220df41 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,132 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
v31-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchtext/x-patch; charset=US-ASCII; name=v31-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patchDownload
From 6583cb0361229af5dbed17d097d73675bbc9b6ee Mon Sep 17 00:00:00 2001
From: Julien Rouhaud <julien.rouhaud@free.fr>
Date: Wed, 11 Dec 2019 13:54:32 +0100
Subject: [PATCH v31 4/5] Add ALTER INDEX ... ALTER COLLATION ... REFRESH
VERSION.
Allow privileged users to declare that the currently installed collation
version, for a specific collation, is binary compatible with the one
that was installed when the index was built. This provides a way to
clear warnings about potentially corrupted indexes without having to use
REINDEX.
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/ref/alter_index.sgml | 17 +++++++
src/backend/catalog/index.c | 2 +-
src/backend/commands/tablecmds.c | 47 +++++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/parser/gram.y | 8 ++++
src/bin/psql/tab-complete.c | 27 ++++++++++-
src/include/catalog/index.h | 3 ++
src/include/nodes/parsenodes.h | 4 +-
.../regress/expected/collate.icu.utf8.out | 20 ++++++++
src/test/regress/sql/collate.icu.utf8.sql | 11 +++++
10 files changed, 137 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 793119d2fc..8e55b681d5 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index a014bfd7c6..69978fb409 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -3181,7 +3181,7 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
-static char *
+char *
index_force_collation_version(const ObjectAddress *otherObject,
const char *version,
void *userdata)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a29c14bf1c..2250f21662 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -559,6 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3986,6 +3988,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4160,6 +4166,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4738,6 +4750,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17582,3 +17599,33 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This overrides an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(coll != NIL);
+ forced_dependency.oid = get_collation_oid(coll, false);
+
+ /* Retrieve the current version for the CURRENT VERSION case. */
+ Assert(OidIsValid(forced_dependency.oid));
+ forced_dependency.version =
+ get_collation_version_for_oid(forced_dependency.oid);
+
+ object.classId = RelationRelationId;
+ object.objectId = rel->rd_id;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ac8b57109c..530aac68a7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3215,6 +3215,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 60cf7242a3..357ab93fb6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2591,6 +2591,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 561fe1dff9..766c513d3e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -820,6 +821,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1715,7 +1730,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1765,6 +1781,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9b4de26514..46d5df1613 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -123,6 +123,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
extern void index_check_collation_versions(Oid relid);
+extern char *index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
extern void index_force_collation_versions(Oid indexid, Oid coll,
char *version);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 32dc7cd5ef..266519096e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1849,7 +1849,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1865,6 +1866,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index a7d7249cfa..eb1ee6a5ba 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -2084,6 +2084,26 @@ SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-------
(0 rows)
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index fbe220df41..c9afb88f7d 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -843,6 +843,17 @@ VACUUM FULL collate_part_1;
SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
--
2.20.1
v31-0005-Doc-Add-Collation-Versions-section.patchtext/x-patch; charset=US-ASCII; name=v31-0005-Doc-Add-Collation-Versions-section.patchDownload
From 1fa864c34289a26cf7acce5f491243242bc59570 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 11 Mar 2020 15:01:23 +1300
Subject: [PATCH v31 5/5] Doc: Add Collation Versions section.
Supply a brief introduction to collation version concepts.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 2745b44417..c450febe00 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ If a collation changes for any reason, persistent data structures such as
+ b-trees that depend on a stable ordering of text might be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording
+ the current version of each referenced collation for any index that
+ depends on it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes it available. If the provider
+ later begins to report a different version, a warning will be issued
+ when the index is accessed, until either the <xref linkend="sql-reindex"/>
+ or the <xref linkend="sql-alterindex"/> command is used to update the
+ version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems) and Windows.
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
--
2.20.1
Re: Collation versioning
On Thu, Oct 22, 2020 at 8:00 PM Thomas Munro <thomas.munro@gmail.com> wrote:
On Thu, Sep 24, 2020 at 9:49 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Sun, Sep 20, 2020 at 10:24:26AM +0800, Julien Rouhaud wrote:
On the other hand the *_pattern_ops are entirely hardcoded, and I
don't think that we'll ever have an extensible way to have this kind
of magic exception. So maybe having a flag at the am level is
acceptable?Hearing no complaint, I kept the flag at the AM level and added hardcoded
exceptions for the *_pattern_ops opclasses to avoid false positive as much as
possible, and no false negative (at least that I'm aware of). I added many
indexes to the regression tests to make sure that all the cases are correctly
handled.Unfortunately, there's still one case that can't be fixed easily. Here's an
example of such case:CREATE INDEX ON sometable ((collatable_col || collatable_col) text_pattern_ops)
I think we should try to get the basic feature into the tree, and then
look at these kinds of subtleties as later improvements.
Agreed.
Here's a new
version with the following changes:1. Update the doc patch to mention that this stuff now works on
Windows too (see commit 352f6f2d).
2. Drop non_deterministic_only argument for from GetTypeCollations();
it was unused.
3. Drop that "stable collation order" field at the AM level for now.
This means that we'll warn you about collation versions changes for
hash and bloom indexes, even when it's technically unnecessary, for
now.The pattern ops stuff seems straightforward however, so let's keep
that bit in the initial commit of the feature. That's a finite set of
hard coded op classes which exist specifically to ignore collations.
Thanks a lot! I'm fine with all the changes. The "stable collation
order" part would definitely benefit from more thoughts, so it's good
if we can focus on that later.
While reviewing the changes, I found a couple of minor issues
(inherited from the previous versions). It's missing a
free_objects_addresses in recordDependencyOnCollations, and there's a
small typo. Inline diff:
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index cab552eb32..4680b4e538 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1674,6 +1674,8 @@ recordDependencyOnCollations(ObjectAddress *myself,
eliminate_duplicate_dependencies(addrs);
recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
DEPENDENCY_NORMAL, record_version);
+
+ free_object_addresses(addrs);
} /*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 69978fb409..048a41f446 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1154,7 +1154,7 @@ index_create(Relation heapRelation,
colls_pattern_ops = list_difference_oid(colls_pattern_ops, colls); /*
- * Record the dependencies for collation declares with any of the
+ * Record the dependencies for collation declared with any of the
* *_pattern_ops opclass, without version tracking.
*/
if (colls_pattern_ops != NIL)Other than that it all looks good to me!
Re: Collation versioning
On Fri, Oct 23, 2020 at 2:07 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
While reviewing the changes, I found a couple of minor issues
(inherited from the previous versions). It's missing a
free_objects_addresses in recordDependencyOnCollations, and there's a
small typo. Inline diff:
Thanks, fixed.
I spent the past few days trying to simplify this patch further.
Here's what I came up with:
1. Drop the function dependencyExists() and related code, which in
earlier versions tried to avoid creating duplicate pg_depends rows by
merging with existing rows. This was rather complicated, and there
are not many duplicates anyway, and it's easier to suppress duplicate
warnings at warning time (see do_collation_version_check()). (I'm not
against working harder to make pg_depend rows unique, but it's not
required for this and I didn't like that way of doing it.)
2. Use index_update_collation_versions() as the workhorse for
REINDEX, binary_upgrade_set_index_coll_version() and ALTER INDEX ...
ALTER COLLATION ... REFRESH_VERSION, instead of having multiple
functions doing similar things. (I wondered about changing the
REINDEX case to simply blow away all dependencies and re-do the
analysis from scratch, which might be useful for future applications
of refobjversion where the set of depended-on objects might change,
but that's out of scope for now and could be added easily enough.)
3. Likewise, drop the function getDependencyVersion() and the
modifications to changeDependenciesOf(); that was yet more catalog
manipulation stuff, but I couldn't see why we don't just call
index_update_collation_versions(newIndexId) after the CREATE INDEX
CONCURRENTLY switcheroo step (see index_concurrently_swap()).
4. Drop track_version from find_expr_references_context, and also
drop the pre-existing code that skipped default collations. I think
it's easier to just let find_expr_references() collect everything, and
leave it to later code to worry about whether to suppress "system
pinned" stuff. It reduces the amount of code that has to know about
refobjversion.
5. General code tidying, pgindent, wordsmithing etc.
I'm not sure what I think about recordMultipleDependencies() being the
function that knows how to fetch the version but I'm not sure if it's
worth the refactoring effort to make ObjectAddresses (essentially a
stretchy vector type) able to carry versions so we can pass that stuff
in.
Attachments:
v32-0001-Remove-pg_collation.collversion.patchtext/x-patch; charset=US-ASCII; name=v32-0001-Remove-pg_collation.collversion.patchDownload
From 0fe4245901e2893747e40bcef2a17ce158aa2dc3 Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Fri, 23 Oct 2020 13:46:04 +1300
Subject: [PATCH v32 1/3] Remove pg_collation.collversion.
This model couldn't be extended to cover the default collation, and
didn't have any information about the affected database objects.
Remove, in preparation for a follow-up commit that will add a new
mechanism.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 -------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 88 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 24 +----
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 7 insertions(+), 343 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 5bd54cb218..d56b3f8170 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2361,17 +2361,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c99499e52b..3b3d7d1c0e 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25481,11 +25481,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index af9ff2867b..65429aabe2 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..5ad8886e60 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -61,14 +61,12 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *lcctypeEl = NULL;
DefElem *providerEl = NULL;
DefElem *deterministicEl = NULL;
- DefElem *versionEl = NULL;
char *collcollate = NULL;
char *collctype = NULL;
char *collproviderstr = NULL;
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -96,8 +94,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
defelp = &providerEl;
else if (strcmp(defel->defname, "deterministic") == 0)
defelp = &deterministicEl;
- else if (strcmp(defel->defname, "version") == 0)
- defelp = &versionEl;
else
{
ereport(ERROR,
@@ -166,9 +162,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +208,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +216,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +266,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +523,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +583,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +644,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..ac8b57109c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3224,16 +3224,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5229,9 +5219,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..0cf90ef33c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3283,9 +3275,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..60cf7242a3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -835,7 +835,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10169,21 +10168,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..f398027fa6 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1842,10 +1842,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2993,10 +2989,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3609,10 +3601,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 07299dbc09..514e0fa0af 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1513,8 +1513,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1616,41 +1614,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ff45e3fb8c..1d09acb64c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13586,12 +13586,10 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
- "collprovider, "
- "collversion, ");
+ "collprovider, ");
else
appendPQExpBufferStr(query,
- "'c' AS collprovider, "
- "NULL AS collversion, ");
+ "'c' AS collprovider, ");
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(query,
@@ -13652,24 +13650,6 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
appendStringLiteralAH(q, collctype, fout);
}
- /*
- * For binary upgrade, carry over the collation version. For normal
- * dump/restore, omit the version, so that it is computed upon restore.
- */
- if (dopt->binary_upgrade)
- {
- int i_collversion;
-
- i_collversion = PQfnumber(res, "collversion");
- if (!PQgetisnull(res, 0, i_collversion))
- {
- appendPQExpBufferStr(q, ", version = ");
- appendStringLiteralAH(q,
- PQgetvalue(res, 0, i_collversion),
- fout);
- }
- }
-
appendPQExpBufferStr(q, ");\n");
if (dopt->binary_upgrade)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..32dc7cd5ef 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1875,17 +1875,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v32-0002-Add-pg_depend.refobjversion.patchtext/x-patch; charset=US-ASCII; name=v32-0002-Add-pg_depend.refobjversion.patchDownload
From 7d28a0bdc36e26e12cffc315bf7d5f32f076507a Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Fri, 23 Oct 2020 13:54:31 +1300
Subject: [PATCH v32 2/3] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. A follow-up commit will use this to record dependencies on
collation versions for indexes, but similar ideas for other kinds of
objects have also been mooted so we want an extensible model.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ++++++
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 61 insertions(+), 32 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index d56b3f8170..ddf4974a9a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3302,6 +3302,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object.
+ </para>
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..1a927377e7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1600,7 +1600,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1687,7 +1689,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1707,7 +1711,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2679,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 454e569fa9..09c30b13e8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -115,6 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the dependency. Note we don't bother to check for duplicate
* dependencies; there's no harm in them.
*/
+ memset(slot[slot_stored_count]->tts_isnull, false,
+ slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+
slot[slot_stored_count]->tts_values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
@@ -122,9 +127,10 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot[slot_stored_count]->tts_values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
-
- memset(slot[slot_stored_count]->tts_isnull, false,
- slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+ if (version)
+ slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ slot[slot_stored_count]->tts_isnull[Anum_pg_depend_refobjversion - 1] = true;
ExecStoreVirtualTuple(slot[slot_stored_count]);
slot_stored_count++;
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index ee3bfa82f4..daf60d0c77 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1539,55 +1539,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v32-0003-Track-collation-versions-for-indexes.patchtext/x-patch; charset=US-ASCII; name=v32-0003-Track-collation-versions-for-indexes.patchDownload
From 7db4ca23cb08c1771a022ecebd38caedd9764cba Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Fri, 23 Oct 2020 14:25:30 +1300
Subject: [PATCH v32 3/3] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match (but only once per
session).
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 3 +-
doc/src/sgml/charset.sgml | 35 +++
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/alter_index.sgml | 17 ++
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 9 +
src/backend/catalog/dependency.c | 210 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 228 +++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 46 +++-
src/backend/catalog/pg_type.c | 60 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/tablecmds.c | 32 +++
src/backend/commands/vacuum.c | 32 +++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/optimizer/util/plancat.c | 9 +
src/backend/parser/gram.y | 8 +
src/backend/utils/adt/pg_locale.c | 51 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++-----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/bin/psql/tab-complete.c | 27 +-
src/include/catalog/dependency.h | 24 +-
src/include/catalog/index.h | 6 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/nodes/parsenodes.h | 4 +-
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 3 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 211 +++++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 137 ++++++++++
src/tools/pgindent/typedefs.list | 2 +
46 files changed, 1640 insertions(+), 154 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index ddf4974a9a..d5e5dd887e 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3308,7 +3308,8 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<structfield>refobjversion</structfield> <type>text</type>
</para>
<para>
- An optional version for the referenced object.
+ An optional version for the referenced object. Currently used for
+ collations (see <xref linkend="collation-versions"/>).
</para>
</entry>
</row>
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 2745b44417..785d2afe9e 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ When a collation changes because of an operating system upgrade,
+ persistent data structures such as B-trees that depend on key order might
+ be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording the
+ current version of each referenced collation for any index that depends on
+ it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes that information available. If the
+ provider later begins to report a different version, a warning will be
+ issued when the index is accessed, until the <xref linkend="sql-reindex"/>
+ or <xref linkend="sql-alterindex"/> command is used to update the version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems) and Windows.
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 3b3d7d1c0e..be90c92e5f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25481,7 +25481,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 793119d2fc..8e55b681d5 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index b59c5697a3..527c847e27 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index fa43e3a972..7ad5e34894 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -38,6 +38,15 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
several scenarios in which to use <command>REINDEX</command>:
<itemizedlist>
+ <listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index.
+ See <xref linkend="collation-versions"/> for more information.
+ </para>
+ </listitem>
+
<listitem>
<para>
An index has become corrupted, and no longer contains valid
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..28278b21be 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,7 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +436,83 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that 'object' depend on. If the function
+ * returns a non-NULL pointer to a new version string, use it to update
+ * refobjversion.
+ */
+void
+visitDependenciesOf(const ObjectAddress *object,
+ VisitDependenciesOfCB callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ new_version = callback(&otherObject,
+ isnull ? NULL : TextDatumGetCString(depversion),
+ userdata);
+
+ /* Does the callback want to update the version? */
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,38 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Record dependencies on a list of collations, optionally with their current
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ eliminate_duplicate_dependencies(addrs);
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+
+ free_object_addresses(addrs);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1699,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1717,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1745,17 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1811,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1833,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1856,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1895,44 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+
+ /*
+ * Otherwise, it may be a composite type having underlying
+ * collations.
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll))
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1957,9 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant.
*/
- if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(con->constcollid))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,8 +2048,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, param->paramtype, 0,
context->addrs);
/* and its collation, just as for Consts */
- if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(param->paramcollid))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,8 +2135,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, fselect->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(fselect->resultcollid))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,8 +2165,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, relab->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(relab->resultcollid))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,8 +2177,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, iocoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(iocoerce->resultcollid))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,8 +2189,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, acoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(acoerce->resultcollid))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,8 +2277,7 @@ find_expr_references_walker(Node *node,
if (OidIsValid(wc->endInRangeFunc))
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
- if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ if (OidIsValid(wc->inRangeColl))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2422,7 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2444,7 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2840,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..fc5140daa6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2336,7 +2336,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2346,7 +2346,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3665,7 +3665,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..4b2fc1d297 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/rel.h"
@@ -1020,6 +1022,8 @@ index_create(Relation heapRelation,
ObjectAddress myself,
referenced;
ObjectAddresses *addrs;
+ List *colls = NIL,
+ *colls_no_version = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1103,30 +1107,65 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* placeholder for normal dependencies */
- addrs = new_object_addresses();
-
- /* Store dependency on collations */
-
- /* The default collation is pinned, so don't bother recording it */
+ /* Get dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- add_exact_object_address(&referenced, addrs);
+ Oid opclass = classObjectId[i];
+
+ /*
+ * The *_pattern_ops opclasses are special: they explicitly do
+ * not depend on collation order so we can save some effort.
+ *
+ * XXX With more analysis, we could also skip version tracking
+ * for some cases like hash indexes with deterministic
+ * collations, because they will never need to order strings.
+ */
+ if (opclass == TEXT_BTREE_PATTERN_OPS_OID ||
+ opclass == VARCHAR_BTREE_PATTERN_OPS_OID ||
+ opclass == BPCHAR_BTREE_PATTERN_OPS_OID)
+ colls_no_version = lappend_oid(colls_no_version, colloid);
+ else
+ colls = lappend_oid(colls, colloid);
+ }
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ /*
+ * Even though there is no top-level collation, there may be
+ * collations affecting ordering inside types, so look there
+ * too.
+ */
+ colls = list_concat(colls, GetTypeCollations(att->atttypid));
}
}
+ /*
+ * If we have anything in both lists, keep just the versioned one to
+ * save some duplication.
+ */
+ if (colls_no_version != NIL && colls != NIL)
+ colls_no_version = list_difference_oid(colls_no_version, colls);
+
+ /* Store the versioned and unversioned collation dependencies. */
+ if (colls_no_version != NIL)
+ recordDependencyOnCollations(&myself, colls_no_version, false);
+ if (colls != NIL)
+ recordDependencyOnCollations(&myself, colls, true);
+
/* Store dependency on operator classes */
+ addrs = new_object_addresses();
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
ObjectAddressSet(referenced, OperatorClassRelationId, classObjectId[i]);
add_exact_object_address(&referenced, addrs);
}
-
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
free_object_addresses(addrs);
@@ -1137,7 +1176,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
/* Store dependencies on anything mentioned in predicate */
@@ -1147,7 +1186,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1226,6 +1265,160 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+typedef struct do_collation_version_check_context
+{
+ Oid relid;
+ List *checked_colls;
+} do_collation_version_check_context;
+
+/*
+ * Raise a warning if the recorded and current collation version don't match.
+ * This is a callback for visitDependenciesOf().
+ */
+static char *
+do_collation_version_check(const ObjectAddress *otherObject,
+ const char *version,
+ void *data)
+{
+ do_collation_version_check_context *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * If we've already checked this collation, skip it. We don't expect too
+ * many duplicates, but it's possible, and we don't want to generate
+ * duplicate warnings.
+ */
+ if (list_member_oid(context->checked_colls, otherObject->objectId))
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ if (!version ||
+ (strcmp(version, "") == 0 && strcmp(current_version, "") != 0))
+ {
+ /*
+ * The collation provider has learned how to report versions, or
+ * the previous version was unknown (pg_upgrade from a release
+ * that didn't record them) and now it is known.
+ */
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(context->relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ /*
+ * The version has changed, probably due to an OS/library upgrade
+ * or streaming replication between different OS/library versions.
+ */
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(context->relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ /* Remember not to complain about this collation again. */
+ context->checked_colls = lappend_oid(context->checked_colls,
+ otherObject->objectId);
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given index.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ do_collation_version_check_context context;
+ ObjectAddress object;
+
+ /*
+ * The callback needs the relid for error messages, and some scratch space
+ * to avoid duplicate warnings.
+ */
+ context.relid = relid;
+ context.checked_colls = NIL;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+
+ visitDependenciesOf(&object, &do_collation_version_check, &context);
+
+ list_free(context.checked_colls);
+}
+
+typedef struct do_collation_version_update_context
+{
+ Oid coll;
+ char *import_version;
+} do_collation_version_update_context;
+
+/*
+ * Update the version for collations. A callback for visitDependenciesOf().
+ */
+static char *
+do_collation_version_update(const ObjectAddress *otherObject,
+ const char *version,
+ void *data)
+{
+ do_collation_version_update_context *context = data;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* If we're looking for a specific collation, skip non-matches. */
+ if (OidIsValid(context->coll) && otherObject->objectId != context->coll)
+ return NULL;
+
+ /* During pg_upgrade, this is used to import the old cluster's version. */
+ if (context->import_version)
+ return context->import_version;
+
+ return get_collation_version_for_oid(otherObject->objectId);
+}
+
+/*
+ * Record the current versions of one or all collations that an index depends
+ * on. InvalidOid means all collations, otherwise only a specific collation's
+ * version dependency is updated. In the latter case, the version can be
+ * passed in, instead of asking the collation provider (for use during
+ * pg_upgrade, to import the version from an old cluster).
+ */
+void
+index_update_collation_versions(Oid relid, Oid coll, char *import_version)
+{
+ do_collation_version_update_context context;
+ ObjectAddress object;
+
+ context.coll = coll;
+ context.import_version = import_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependenciesOf(&object, &do_collation_version_update, &context);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1686,6 +1879,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ /* Now we have the old index's collation versions, so fix that. */
+ CommandCounterIncrement();
+ index_update_collation_versions(newIndexId, InvalidOid, NULL);
+
/*
* Copy over statistics from old to new index
*/
@@ -3635,6 +3832,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId, InvalidOid, NULL);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 0d70cb0c3c..93774c9d21 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -362,7 +362,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 09c30b13e8..01bd17599a 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,6 +28,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
@@ -45,19 +47,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record is added even if the referenced
+ * object is pinned, and the dependency version will be retrieved according to
+ * the referenced object kind. For now, only collation version is
+ * supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -66,6 +73,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
max_slots,
slot_init_count,
slot_stored_count;
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -96,12 +104,38 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* For now we only know how to deal with collations. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX don't need version tracking. */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ version = get_collation_version_for_oid(referenced->objectId);
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (isObjectPinned(referenced, dependDesc))
+ if (!ignore_systempin && isObjectPinned(referenced, dependDesc))
continue;
if (slot_init_count < max_slots)
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 0b04dff773..44eed1a0b3 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,65 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/*
+ * Get a list of all distinct collations that the given type depends on.
+ */
+List *
+GetTypeCollations(Oid typeoid)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result, GetTypeCollations(rangeid));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a29c14bf1c..1aa3ead9e0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -559,6 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3986,6 +3988,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4160,6 +4166,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4738,6 +4750,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17582,3 +17599,18 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This overrides an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ index_update_collation_versions(rel->rd_id,
+ get_collation_oid(coll, false),
+ NULL);
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ddeec870d8..e1016609dc 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ac8b57109c..530aac68a7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3215,6 +3215,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index f9d0d67aa7..3a20dcdee5 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 60cf7242a3..357ab93fb6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2591,6 +2591,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..faa9c7f533 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,50 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string for a given collation OID.
+ *
+ * An empty string is returned when the version is unknown. Note that NULL
+ * means something different ("untracked"), when stored in refobjversion.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..0950e3b4fc 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_update_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9061af81a3..153adc157d 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5934,6 +5935,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..a1151e3b31 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -179,6 +179,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1d09acb64c..b82c16b151 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +388,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -712,6 +716,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7028,7 +7036,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7064,7 +7074,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7089,7 +7154,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7128,7 +7195,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7163,7 +7232,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7194,7 +7265,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7228,7 +7301,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7268,6 +7343,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7293,6 +7370,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16359,7 +16438,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16426,6 +16506,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16454,6 +16538,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18438,6 +18537,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..623814d1c5 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if a partition, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b2b4f1fd4d..912eb7409f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -820,6 +821,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1715,7 +1730,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1765,6 +1781,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..08834194d5 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -160,7 +160,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +181,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependenciesOfCB) (const ObjectAddress *otherObject,
+ const char *version,
+ void *data);
+
+extern void visitDependenciesOf(const ObjectAddress *object,
+ VisitDependenciesOfCB callback,
+ void *data);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +222,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..3e205ab068 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +133,10 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid,
+ Oid coll,
+ char *import_version);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index bbcac69d48..7abfa4d65d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10387,6 +10387,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..f3061bca57 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 32dc7cd5ef..266519096e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1849,7 +1849,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1865,6 +1866,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..c5ffea40f2 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check been done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index 9774f534d9..14cde4f5ba 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index ebcaeb44fe..c1b5d4aabd 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -769,10 +769,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..da5a03663d 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,217 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+-----------------------------------+------------+---------------------
+ icuidx00_val | "fr-x-icu" | up to date
+ icuidx00_val_val | "fr-x-icu" | up to date
+ icuidx00_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val | "fr-x-icu" | up to date
+ icuidx00_val_val_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_where | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_where | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(63 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..6cb7786e13 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,14 +2065,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2092,14 +2094,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c9afb88f7d 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,143 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
-- cleanup
RESET search_path;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index ff853634bc..da3e5f73d0 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2913,6 +2913,8 @@ dlist_head
dlist_iter
dlist_mutable_iter
dlist_node
+do_collation_version_check_context
+do_collation_version_update_context
ds_state
dsa_area
dsa_area_control
--
2.20.1
Re: Collation versioning
On Sun, Oct 25, 2020 at 10:36 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Fri, Oct 23, 2020 at 2:07 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
While reviewing the changes, I found a couple of minor issues
(inherited from the previous versions). It's missing a
free_objects_addresses in recordDependencyOnCollations, and there's a
small typo. Inline diff:Thanks, fixed.
I spent the past few days trying to simplify this patch further.
Here's what I came up with:
Thanks!
1. Drop the function dependencyExists() and related code, which in
earlier versions tried to avoid creating duplicate pg_depends rows by
merging with existing rows. This was rather complicated, and there
are not many duplicates anyway, and it's easier to suppress duplicate
warnings at warning time (see do_collation_version_check()). (I'm not
against working harder to make pg_depend rows unique, but it's not
required for this and I didn't like that way of doing it.)
I didn't review all the changes yet, so I'll probably post a deeper
review tomorrow. I'm not opposed to this new approach, as it indeed
saves a lot of code. However, looking at
do_collation_version_check(), it seems that you're saving the
collation in context->checked_calls even if it didn't raise a WARNING.
Since you can now have indexes with dependencies on a same collation
with both version tracked and untracked (see for instance
icuidx00_val_pattern_where in the regression tests), can't this hide
corruption warning reports if the untracked version is found first?
That can be easily fixed, so no objection to that approach of course.
Re: Collation versioning
On Sun, Oct 25, 2020 at 7:13 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Sun, Oct 25, 2020 at 10:36 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Fri, Oct 23, 2020 at 2:07 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
While reviewing the changes, I found a couple of minor issues
(inherited from the previous versions). It's missing a
free_objects_addresses in recordDependencyOnCollations, and there's a
small typo. Inline diff:Thanks, fixed.
I spent the past few days trying to simplify this patch further.
Here's what I came up with:Thanks!
1. Drop the function dependencyExists() and related code, which in
earlier versions tried to avoid creating duplicate pg_depends rows by
merging with existing rows. This was rather complicated, and there
are not many duplicates anyway, and it's easier to suppress duplicate
warnings at warning time (see do_collation_version_check()). (I'm not
against working harder to make pg_depend rows unique, but it's not
required for this and I didn't like that way of doing it.)I didn't review all the changes yet, so I'll probably post a deeper
review tomorrow. I'm not opposed to this new approach, as it indeed
saves a lot of code. However, looking at
do_collation_version_check(), it seems that you're saving the
collation in context->checked_calls even if it didn't raise a WARNING.
Since you can now have indexes with dependencies on a same collation
with both version tracked and untracked (see for instance
icuidx00_val_pattern_where in the regression tests), can't this hide
corruption warning reports if the untracked version is found first?
That can be easily fixed, so no objection to that approach of course.
I finish looking at the rest of the patches. I don't have much to
say, it all looks good and I quite like how much useless code you got
rid of!
Re: Collation versioning
On Tue, Oct 27, 2020 at 1:34 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Sun, Oct 25, 2020 at 7:13 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
I didn't review all the changes yet, so I'll probably post a deeper
review tomorrow. I'm not opposed to this new approach, as it indeed
saves a lot of code. However, looking at
do_collation_version_check(), it seems that you're saving the
collation in context->checked_calls even if it didn't raise a WARNING.
Since you can now have indexes with dependencies on a same collation
with both version tracked and untracked (see for instance
icuidx00_val_pattern_where in the regression tests), can't this hide
corruption warning reports if the untracked version is found first?
That can be easily fixed, so no objection to that approach of course.
Right, fixed.
I finish looking at the rest of the patches. I don't have much to
say, it all looks good and I quite like how much useless code you got
rid of!
Thanks! I tested a bunch of permutations[1]https://github.com/macdice/some_pg_upgrade_tests of cross-version
pg_update, with and without ICU, with and without libc version
support, and fixed some problems I found in pg_dump:
1. We need to print OIDs as %u, not %d. Also, let's use
'%u'::pg_catalog.oid to be consistent with nearby things.
2. We dump binary_upgrade_set_index_coll_version(<index>, NULL, ...)
to blow away the new cluster's versions before we import the old
versions. OK, but the function was marked STRICT...
3. We dump binary_upgrade_set_index_coll_version(<index>,
<collation>, <version>), to import the old cluster's version, where
<collation> is an OID. OK, but we need the new cluster's OID, not the
old one, so it needs to be an expression like
'pg_catalog."fr_FR"'::regcollation (compare the other references to
collations in the dump, which are all by name).
4. I didn't really like the use of '' for unknown. I figured out how
to use NULL for that.
Attachments:
v33-0001-Remove-pg_collation.collversion.patchtext/x-patch; charset=US-ASCII; name=v33-0001-Remove-pg_collation.collversion.patchDownload
From b18f5de66dfc4b21e84a1947ba3a18dd5a564146 Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Fri, 23 Oct 2020 13:46:04 +1300
Subject: [PATCH v33 1/3] Remove pg_collation.collversion.
This model couldn't be extended to cover the default collation, and
didn't have any information about the affected database objects.
Remove, in preparation for a follow-up commit that will add a new
mechanism.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 -------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 88 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 24 +----
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 7 insertions(+), 343 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 5bd54cb218..d56b3f8170 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2361,17 +2361,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d8eee3a826..0398b4909b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25444,11 +25444,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index af9ff2867b..65429aabe2 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..5ad8886e60 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -61,14 +61,12 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *lcctypeEl = NULL;
DefElem *providerEl = NULL;
DefElem *deterministicEl = NULL;
- DefElem *versionEl = NULL;
char *collcollate = NULL;
char *collctype = NULL;
char *collproviderstr = NULL;
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -96,8 +94,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
defelp = &providerEl;
else if (strcmp(defel->defname, "deterministic") == 0)
defelp = &deterministicEl;
- else if (strcmp(defel->defname, "version") == 0)
- defelp = &versionEl;
else
{
ereport(ERROR,
@@ -166,9 +162,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +208,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +216,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +266,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +523,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +583,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +644,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..ac8b57109c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3224,16 +3224,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5229,9 +5219,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..0cf90ef33c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3283,9 +3275,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..60cf7242a3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -835,7 +835,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10169,21 +10168,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..f398027fa6 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1842,10 +1842,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2993,10 +2989,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3609,10 +3601,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 07299dbc09..514e0fa0af 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1513,8 +1513,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1616,41 +1614,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 03f9d4d9e8..9851022a53 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13589,12 +13589,10 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
- "collprovider, "
- "collversion, ");
+ "collprovider, ");
else
appendPQExpBufferStr(query,
- "'c' AS collprovider, "
- "NULL AS collversion, ");
+ "'c' AS collprovider, ");
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(query,
@@ -13655,24 +13653,6 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
appendStringLiteralAH(q, collctype, fout);
}
- /*
- * For binary upgrade, carry over the collation version. For normal
- * dump/restore, omit the version, so that it is computed upon restore.
- */
- if (dopt->binary_upgrade)
- {
- int i_collversion;
-
- i_collversion = PQfnumber(res, "collversion");
- if (!PQgetisnull(res, 0, i_collversion))
- {
- appendPQExpBufferStr(q, ", version = ");
- appendStringLiteralAH(q,
- PQgetvalue(res, 0, i_collversion),
- fout);
- }
- }
-
appendPQExpBufferStr(q, ");\n");
if (dopt->binary_upgrade)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ff584f2955..319f77013f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1879,17 +1879,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v33-0002-Add-pg_depend.refobjversion.patchtext/x-patch; charset=US-ASCII; name=v33-0002-Add-pg_depend.refobjversion.patchDownload
From 80fa6acb80e5adf48058ab7f0484901d0e5d42f5 Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Fri, 23 Oct 2020 13:54:31 +1300
Subject: [PATCH v33 2/3] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. A follow-up commit will use this to record dependencies on
collation versions for indexes, but similar ideas for other kinds of
objects have also been mooted so we want an extensible model.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ++++++
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 61 insertions(+), 32 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index d56b3f8170..ddf4974a9a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3302,6 +3302,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object.
+ </para>
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..1a927377e7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1600,7 +1600,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1687,7 +1689,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1707,7 +1711,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2679,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 454e569fa9..09c30b13e8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -115,6 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the dependency. Note we don't bother to check for duplicate
* dependencies; there's no harm in them.
*/
+ memset(slot[slot_stored_count]->tts_isnull, false,
+ slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+
slot[slot_stored_count]->tts_values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
@@ -122,9 +127,10 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot[slot_stored_count]->tts_values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
-
- memset(slot[slot_stored_count]->tts_isnull, false,
- slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+ if (version)
+ slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ slot[slot_stored_count]->tts_isnull[Anum_pg_depend_refobjversion - 1] = true;
ExecStoreVirtualTuple(slot[slot_stored_count]);
slot_stored_count++;
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index ee3bfa82f4..daf60d0c77 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1539,55 +1539,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v33-0003-Track-collation-versions-for-indexes.patchtext/x-patch; charset=US-ASCII; name=v33-0003-Track-collation-versions-for-indexes.patchDownload
From 2a121f754d98b5d5647971b2b25f7ceb9d9325aa Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Fri, 23 Oct 2020 14:25:30 +1300
Subject: [PATCH v33 3/3] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match (but only once per
session).
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 3 +-
doc/src/sgml/charset.sgml | 35 +++
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/alter_index.sgml | 17 ++
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 9 +
src/backend/catalog/dependency.c | 211 ++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 273 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 46 ++-
src/backend/catalog/pg_type.c | 60 ++++
src/backend/commands/collationcmds.c | 16 +-
src/backend/commands/tablecmds.c | 31 ++
src/backend/commands/vacuum.c | 32 ++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/optimizer/util/plancat.c | 9 +
src/backend/parser/gram.y | 8 +
src/backend/utils/adt/pg_locale.c | 46 ++-
src/backend/utils/adt/pg_upgrade_support.c | 27 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 177 +++++++++++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/pg_dump/t/002_pg_dump.pl | 254 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/bin/psql/tab-complete.c | 27 +-
src/include/catalog/dependency.h | 25 +-
src/include/catalog/index.h | 6 +
src/include/catalog/pg_proc.dat | 5 +
src/include/catalog/pg_type.h | 2 +
src/include/nodes/parsenodes.h | 4 +-
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 3 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 67 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 211 ++++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 137 +++++++++
src/tools/pgindent/typedefs.list | 2 +
46 files changed, 1669 insertions(+), 150 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index ddf4974a9a..d5e5dd887e 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3308,7 +3308,8 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<structfield>refobjversion</structfield> <type>text</type>
</para>
<para>
- An optional version for the referenced object.
+ An optional version for the referenced object. Currently used for
+ collations (see <xref linkend="collation-versions"/>).
</para>
</entry>
</row>
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 2745b44417..785d2afe9e 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ When a collation changes because of an operating system upgrade,
+ persistent data structures such as B-trees that depend on key order might
+ be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording the
+ current version of each referenced collation for any index that depends on
+ it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes that information available. If the
+ provider later begins to report a different version, a warning will be
+ issued when the index is accessed, until the <xref linkend="sql-reindex"/>
+ or <xref linkend="sql-alterindex"/> command is used to update the version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems) and Windows.
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 0398b4909b..8185898b0c 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25444,7 +25444,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. <literal>null</literal> is returned
+ if the version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 793119d2fc..8e55b681d5 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index b59c5697a3..527c847e27 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index fa43e3a972..7ad5e34894 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -38,6 +38,15 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
several scenarios in which to use <command>REINDEX</command>:
<itemizedlist>
+ <listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index.
+ See <xref linkend="collation-versions"/> for more information.
+ </para>
+ </listitem>
+
<listitem>
<para>
An index has become corrupted, and no longer contains valid
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..c84a3e03d8 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,7 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +436,84 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that 'object' depend on. If the function
+ * returns true, refobjversion will be updated in the catalog.
+ */
+void
+visitDependenciesOf(const ObjectAddress *object,
+ VisitDependenciesOfCB callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ /* Does the callback want to update the version? */
+ if (callback(&otherObject,
+ isnull ? NULL : TextDatumGetCString(depversion),
+ &new_version,
+ userdata))
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ if (new_version)
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1646,38 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Record dependencies on a list of collations, optionally with their current
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ eliminate_duplicate_dependencies(addrs);
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+
+ free_object_addresses(addrs);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1700,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1718,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1746,17 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1812,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1834,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1857,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1896,44 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+
+ /*
+ * Otherwise, it may be a composite type having underlying
+ * collations.
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll))
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1958,9 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant.
*/
- if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(con->constcollid))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,8 +2049,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, param->paramtype, 0,
context->addrs);
/* and its collation, just as for Consts */
- if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(param->paramcollid))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,8 +2136,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, fselect->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(fselect->resultcollid))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,8 +2166,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, relab->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(relab->resultcollid))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,8 +2178,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, iocoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(iocoerce->resultcollid))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,8 +2190,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, acoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(acoerce->resultcollid))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,8 +2278,7 @@ find_expr_references_walker(Node *node,
if (OidIsValid(wc->endInRangeFunc))
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
- if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ if (OidIsValid(wc->inRangeColl))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2423,7 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2445,7 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2841,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..fc5140daa6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2336,7 +2336,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2346,7 +2346,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3665,7 +3665,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..4dfb5ed32a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/rel.h"
@@ -1020,6 +1022,8 @@ index_create(Relation heapRelation,
ObjectAddress myself,
referenced;
ObjectAddresses *addrs;
+ List *colls = NIL,
+ *colls_no_version = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1103,30 +1107,65 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* placeholder for normal dependencies */
- addrs = new_object_addresses();
-
- /* Store dependency on collations */
-
- /* The default collation is pinned, so don't bother recording it */
+ /* Get dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- add_exact_object_address(&referenced, addrs);
+ Oid opclass = classObjectId[i];
+
+ /*
+ * The *_pattern_ops opclasses are special: they explicitly do
+ * not depend on collation order so we can save some effort.
+ *
+ * XXX With more analysis, we could also skip version tracking
+ * for some cases like hash indexes with deterministic
+ * collations, because they will never need to order strings.
+ */
+ if (opclass == TEXT_BTREE_PATTERN_OPS_OID ||
+ opclass == VARCHAR_BTREE_PATTERN_OPS_OID ||
+ opclass == BPCHAR_BTREE_PATTERN_OPS_OID)
+ colls_no_version = lappend_oid(colls_no_version, colloid);
+ else
+ colls = lappend_oid(colls, colloid);
+ }
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ /*
+ * Even though there is no top-level collation, there may be
+ * collations affecting ordering inside types, so look there
+ * too.
+ */
+ colls = list_concat(colls, GetTypeCollations(att->atttypid));
}
}
+ /*
+ * If we have anything in both lists, keep just the versioned one to
+ * save some duplication.
+ */
+ if (colls_no_version != NIL && colls != NIL)
+ colls_no_version = list_difference_oid(colls_no_version, colls);
+
+ /* Store the versioned and unversioned collation dependencies. */
+ if (colls_no_version != NIL)
+ recordDependencyOnCollations(&myself, colls_no_version, false);
+ if (colls != NIL)
+ recordDependencyOnCollations(&myself, colls, true);
+
/* Store dependency on operator classes */
+ addrs = new_object_addresses();
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
ObjectAddressSet(referenced, OperatorClassRelationId, classObjectId[i]);
add_exact_object_address(&referenced, addrs);
}
-
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
free_object_addresses(addrs);
@@ -1137,7 +1176,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
/* Store dependencies on anything mentioned in predicate */
@@ -1147,7 +1186,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1226,6 +1265,205 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+typedef struct do_collation_version_check_context
+{
+ Oid relid;
+ List *unknown_version_colls;
+ List *checked_colls;
+} do_collation_version_check_context;
+
+/*
+ * Raise a warning if the recorded and current collation version don't match.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_collation_version_check(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ do_collation_version_check_context *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ if (!current_version)
+ return false;
+
+ /*
+ * If no version was recorded, we'll consider emitting a warning at the end
+ * of processing, but only if we don't see a versioned record for the same
+ * collation.
+ */
+ if (!version)
+ {
+ context->unknown_version_colls =
+ list_append_unique_oid(context->unknown_version_colls,
+ otherObject->objectId);
+ return false;
+ }
+
+ /*
+ * If we've already checked this collation, skip it. We don't expect too
+ * many duplicates, but it's possible, and we don't want to generate
+ * duplicate warnings. We don't expect their versions to differ.
+ */
+ if (list_member_oid(context->checked_colls, otherObject->objectId))
+ return false;
+
+ if (strcmp(current_version, version) != 0)
+ {
+ /*
+ * The version has changed, probably due to an OS/library upgrade or
+ * streaming replication between different OS/library versions.
+ */
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(context->relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+
+ /* Remember not to complain about this collation again. */
+ context->checked_colls = lappend_oid(context->checked_colls,
+ otherObject->objectId);
+
+ return false;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given index.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ do_collation_version_check_context context;
+ ObjectAddress object;
+ ListCell *lc;
+
+ /*
+ * The callback needs the relid for error messages, and some scratch space
+ * to avoid duplicate warnings.
+ */
+ context.relid = relid;
+ context.unknown_version_colls = NIL;
+ context.checked_colls = NIL;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+
+ visitDependenciesOf(&object, &do_collation_version_check, &context);
+
+ /* Warn about dependencies with no recorded version. */
+ foreach(lc, context.unknown_version_colls)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ /* If we also saw a record with a version, nothing to do. */
+ if (list_member_oid(context.checked_colls, coll))
+ continue;
+
+ /*
+ * Raise a warning that is worded slightly differently, because we
+ * don't even have a refobjversion value. Expected when upgrading from
+ * a release that didn't track version.
+ */
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(coll),
+ get_collation_version_for_oid(coll)),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+
+ list_free(context.unknown_version_colls);
+ list_free(context.checked_colls);
+}
+
+typedef struct do_collation_version_update_context
+{
+ Oid coll;
+ char *import_version;
+ bool binary_upgrade;
+} do_collation_version_update_context;
+
+/*
+ * Update the version for collations. A callback for visitDependenciesOf().
+ */
+static bool
+do_collation_version_update(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ do_collation_version_update_context *context = data;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return false;
+
+ /* If we're only trying to update one collation, skip others. */
+ if (OidIsValid(context->coll) && otherObject->objectId != context->coll)
+ return false;
+
+ /* When reached by pg_upgrade, import the version. */
+ if (context->binary_upgrade)
+ *new_version = context->import_version;
+ else
+ *new_version = get_collation_version_for_oid(otherObject->objectId);
+
+ return true;
+}
+
+/*
+ * Record the current versions of one or all collations that an index depends
+ * on.
+ */
+void
+index_update_collation_versions(Oid relid, Oid coll)
+{
+ do_collation_version_update_context context;
+ ObjectAddress object;
+
+ context.coll = coll;
+ context.import_version = NULL;
+ context.binary_upgrade = false;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependenciesOf(&object, &do_collation_version_update, &context);
+}
+
+/*
+ * Perform version updates for binary upgrades.
+ */
+void
+index_import_collation_versions(Oid relid, Oid coll, char *import_version)
+{
+ do_collation_version_update_context context;
+ ObjectAddress object;
+
+ context.coll = coll;
+ context.import_version = import_version;
+ context.binary_upgrade = true;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependenciesOf(&object, &do_collation_version_update, &context);
+}
+
+
/*
* index_concurrently_create_copy
*
@@ -1686,6 +1924,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ /* Now we have the old index's collation versions, so fix that. */
+ CommandCounterIncrement();
+ index_update_collation_versions(newIndexId, InvalidOid);
+
/*
* Copy over statistics from old to new index
*/
@@ -3635,6 +3877,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId, InvalidOid);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 0d70cb0c3c..93774c9d21 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -362,7 +362,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 09c30b13e8..01bd17599a 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,6 +28,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
@@ -45,19 +47,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record is added even if the referenced
+ * object is pinned, and the dependency version will be retrieved according to
+ * the referenced object kind. For now, only collation version is
+ * supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -66,6 +73,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
max_slots,
slot_init_count,
slot_stored_count;
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -96,12 +104,38 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* For now we only know how to deal with collations. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX don't need version tracking. */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ version = get_collation_version_for_oid(referenced->objectId);
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (isObjectPinned(referenced, dependDesc))
+ if (!ignore_systempin && isObjectPinned(referenced, dependDesc))
continue;
if (slot_init_count < max_slots)
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 0b04dff773..44eed1a0b3 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,65 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/*
+ * Get a list of all distinct collations that the given type depends on.
+ */
+List *
+GetTypeCollations(Oid typeoid)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result, GetTypeCollations(rangeid));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..7b31272734 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,23 +270,9 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
if (version)
PG_RETURN_TEXT_P(cstring_to_text(version));
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a29c14bf1c..84cda0f24b 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -559,6 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3986,6 +3988,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4160,6 +4166,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4738,6 +4750,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17582,3 +17599,17 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This overrides an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ index_update_collation_versions(rel->rd_id,
+ get_collation_oid(coll, false));
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 1b6717f727..b1423a0444 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ac8b57109c..530aac68a7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3215,6 +3215,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 4f0da51c26..52c01eb86b 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 60cf7242a3..357ab93fb6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2591,6 +2591,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..3b0324ce18 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,45 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string for a given collation OID.
+ * Return NULL if the provider doesn't support versions.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..5b0f7b2d92 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,29 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ if (PG_ARGISNULL(2))
+ version = NULL;
+ else
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_import_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9224e2ffed..66393becfb 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5934,6 +5935,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 6c79319164..1110ab4780 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -179,6 +179,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9851022a53..c269cfe081 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,9 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat,
+ Archive *fount);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -712,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7031,7 +7040,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependcollnames,
+ i_inddependcollversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7067,7 +7078,64 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(quote_ident(ns.nspname) || '.' || quote_ident(c.collname) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend d "
+ " JOIN pg_catalog.pg_collation c ON (c.oid = d.refobjid) "
+ " JOIN pg_catalog.pg_namespace ns ON (c.collnamespace = ns.oid) "
+ " WHERE d.classid = " CppAsString2(RelationRelationId) " AND "
+ " d.objid = i.indexrelid AND "
+ " d.objsubid = 0 AND "
+ " d.refclassid = " CppAsString2(CollationRelationId) " AND "
+ " d.refobjversion IS NOT NULL) AS inddependcollnames, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependcollversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7092,7 +7160,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7131,7 +7201,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7166,7 +7238,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7197,7 +7271,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7231,7 +7307,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7271,6 +7349,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependcollnames = PQfnumber(res, "inddependcollnames");
+ i_inddependcollversions = PQfnumber(res, "inddependcollversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7296,6 +7376,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependcollnames = pg_strdup(PQgetvalue(res, j, i_inddependcollnames));
+ indxinfo[j].inddependcollversions = pg_strdup(PQgetvalue(res, j, i_inddependcollversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16362,7 +16444,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16429,6 +16512,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat, fout);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16457,6 +16544,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat, fout);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18441,6 +18543,65 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Generate calls to binary_update_set_index_coll_version() to import the
+ * collation versions into the new database.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat, Archive *fout)
+{
+ char *inddependcollnames = indxinfo->inddependcollnames;
+ char *inddependcollversions = indxinfo->inddependcollversions;
+ char **inddependcollnamesarray;
+ char **inddependcollversionsarray;
+ int ninddependcollnames;
+ int ninddependcollversions;
+
+ /*
+ * A new cluster's index will have pg_depends rows that captured the
+ * *current* collation versions. Leave those as defaults if the user said
+ * --unknown-collations-binary-compatible, otherwise clear them.
+ */
+ if (!unknown_coll_compat)
+ {
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, clobber new index's collation versions\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version('%u'::pg_catalog.oid, NULL, NULL);\n",
+ indxinfo->dobj.catId.oid);
+ }
+
+ /* Restore the versions that were recorded by the old cluster (if any). */
+ parsePGArray(inddependcollnames,
+ &inddependcollnamesarray,
+ &ninddependcollnames);
+ parsePGArray(inddependcollversions,
+ &inddependcollversionsarray,
+ &ninddependcollversions);
+ Assert(ninddependcollnames == ninddependcollversions);
+
+ if (ninddependcollnames > 0)
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore old index's collation versions\n");
+ for (int i = 0; i < ninddependcollnames; i++)
+ {
+ /*
+ * The OIDs of collations are not preserved, so we need to ask the new
+ * cluster to resolve the name to an OID.
+ */
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version('%u'::pg_catalog.oid, ",
+ indxinfo->dobj.catId.oid);
+ appendStringLiteralAH(buffer,inddependcollnamesarray[i], fout);
+ appendPQExpBuffer(buffer, "::regcollation, %s);\n",
+ inddependcollversionsarray[i]);
+ }
+
+ if (inddependcollnamesarray)
+ free(inddependcollnamesarray);
+ if (inddependcollversionsarray)
+ free(inddependcollversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..317bb83970 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,8 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependcollnames; /* FQ names of depended-on collations */
+ char *inddependcollversions; /* versions of the above */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if a partition, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..3106ec34e1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,59 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ 'binary_upgrade_set_index_coll_version clobber and restore' => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, clobber new index's collation versions\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E\('\d+'::pg_catalog.oid,\ NULL,\ NULL\);\n
+ \n
+ \Q-- For binary upgrade, restore old index's collation versions\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E\('\d+'::pg_catalog.oid,\ '[^']+'::regcollation,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1 },
+ icu => 1,
+ },
+
+ 'binary_upgrade_set_index_coll_version restore only' => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_col2(id integer, val text);
+ CREATE INDEX regress_coll_idx2 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx2\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx2 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore old index's collation versions\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E\('\d+'::pg_catalog.oid,\ '[^']+'::regcollation,\ 'not_a_version'\);/xm,
+ like => { binary_coll_compatible => 1 },
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3411,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3465,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3535,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3594,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b2b4f1fd4d..912eb7409f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -820,6 +821,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1715,7 +1730,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1765,6 +1781,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..c4ed5740b6 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -160,7 +160,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +181,30 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef bool(*VisitDependenciesOfCB) (const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data);
+
+extern void visitDependenciesOf(const ObjectAddress *object,
+ VisitDependenciesOfCB callback,
+ void *data);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +223,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..fdffda227d 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +133,10 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid, Oid coll);
+extern void index_import_collation_versions(Oid relid, Oid coll,
+ char *import_version);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d9770bbadd..0f5de5041d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10386,6 +10386,11 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version',
+ proisstrict => 'f' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..f3061bca57 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 319f77013f..e1aeea2560 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1853,7 +1853,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1869,6 +1870,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..c5ffea40f2 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check been done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index 9774f534d9..14cde4f5ba 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..3f57632b5d
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,67 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 9;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index ebcaeb44fe..c1b5d4aabd 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -769,10 +769,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..da5a03663d 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,217 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+-----------------------------------+------------+---------------------
+ icuidx00_val | "fr-x-icu" | up to date
+ icuidx00_val_val | "fr-x-icu" | up to date
+ icuidx00_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val | "fr-x-icu" | up to date
+ icuidx00_val_val_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_where | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_where | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(63 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..6cb7786e13 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,14 +2065,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2092,14 +2094,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c9afb88f7d 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,143 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
-- cleanup
RESET search_path;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b6acade6c6..03c4e0fe5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2914,6 +2914,8 @@ dlist_head
dlist_iter
dlist_mutable_iter
dlist_node
+do_collation_version_check_context
+do_collation_version_update_context
ds_state
dsa_area
dsa_area_control
--
2.20.1
Re: Collation versioning
On Fri, Oct 30, 2020 at 1:20 AM Thomas Munro <thomas.munro@gmail.com> wrote:
4. I didn't really like the use of '' for unknown. I figured out how
to use NULL for that.
Hrmph. That logic didn't work for the pattern ops case. I fixed
that, by partially reintroducing a special value, but this time
restricting the code that knows about that to just pg_dump, and I used
the value 'unknown', so really it's not special at all as far as the
server is concerned and there is only one kind of warning message.
Furthermore, I realised that I really don't like the policy of
assuming that all text-related indexes imported from older releases
need the "unknown" warning. That'll just make this feature
unnecessarily noisy and unpopular when 14 comes out, essentially
crying wolf, even though it's technically true that the collations in
imported-from-before-14 indexes are of unknown version. Even worse,
instructions might be shared around the internet to show how to shut
the warnings up without reindexing, and then later when there really
is a version change, someone might find those instructions and follow
them! So I propose that the default should be to assume that indexes
are not corrupted, unless you opt into the more pessimistic policy
with --index-collation-versions-unknown. Done like that.
I also realised that I don't like carrying a bunch of C code to
support binary upgrades, when it's really just a hand-coded trivial
UPDATE of pg_depend. Is there any reason pg_dump --binary-upgrade
shouldn't just dump UPDATE statements, and make this whole feature a
lot less mysterious, and shorter? Done like that.
While testing on another OS that will be encountered in the build farm
when I commit this, I realised that I needed to add --encoding=UTF8 to
tests under src/bin/pg_dump and src/test/locale, because they now do
things with ICU collations (if built with ICU support) and that only
works with UTF8. Another option would be to find a way to skip those
tests if the encoding is not UTF8. Hmm, I wonder if it's bad to
effectively remove the testing that comes for free from buildfarm
animals running this under non-UTF8 encodings; but if we actually
valued that, I suppose we'd do it explicitly as another test pass with
SQL_ASCII.
Attachments:
v34-0001-Remove-pg_collation.collversion.patchtext/x-patch; charset=US-ASCII; name=v34-0001-Remove-pg_collation.collversion.patchDownload
From 212ebea7a7d2657a8a6d8ddec4b7be613b1f84f7 Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Fri, 23 Oct 2020 13:46:04 +1300
Subject: [PATCH v34 1/3] Remove pg_collation.collversion.
This model couldn't be extended to cover the default collation, and
didn't have any information about the affected database objects.
Remove, in preparation for a follow-up commit that will add a new
mechanism.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 -------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 88 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 24 +----
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
21 files changed, 7 insertions(+), 343 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 0ccdff1cda..d0d2598290 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2361,17 +2361,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d8eee3a826..0398b4909b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25444,11 +25444,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index af9ff2867b..65429aabe2 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..5ad8886e60 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -61,14 +61,12 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *lcctypeEl = NULL;
DefElem *providerEl = NULL;
DefElem *deterministicEl = NULL;
- DefElem *versionEl = NULL;
char *collcollate = NULL;
char *collctype = NULL;
char *collproviderstr = NULL;
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -96,8 +94,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
defelp = &providerEl;
else if (strcmp(defel->defname, "deterministic") == 0)
defelp = &deterministicEl;
- else if (strcmp(defel->defname, "version") == 0)
- defelp = &versionEl;
else
{
ereport(ERROR,
@@ -166,9 +162,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +208,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +216,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +266,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +523,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +583,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +644,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..ac8b57109c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3224,16 +3224,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5229,9 +5219,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..0cf90ef33c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3283,9 +3275,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..60cf7242a3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -835,7 +835,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10169,21 +10168,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..f398027fa6 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1842,10 +1842,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2993,10 +2989,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3609,10 +3601,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 07299dbc09..514e0fa0af 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1513,8 +1513,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1616,41 +1614,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 03f9d4d9e8..9851022a53 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13589,12 +13589,10 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
- "collprovider, "
- "collversion, ");
+ "collprovider, ");
else
appendPQExpBufferStr(query,
- "'c' AS collprovider, "
- "NULL AS collversion, ");
+ "'c' AS collprovider, ");
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(query,
@@ -13655,24 +13653,6 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
appendStringLiteralAH(q, collctype, fout);
}
- /*
- * For binary upgrade, carry over the collation version. For normal
- * dump/restore, omit the version, so that it is computed upon restore.
- */
- if (dopt->binary_upgrade)
- {
- int i_collversion;
-
- i_collversion = PQfnumber(res, "collversion");
- if (!PQgetisnull(res, 0, i_collversion))
- {
- appendPQExpBufferStr(q, ", version = ");
- appendStringLiteralAH(q,
- PQgetvalue(res, 0, i_collversion),
- fout);
- }
- }
-
appendPQExpBufferStr(q, ");\n");
if (dopt->binary_upgrade)
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ff584f2955..319f77013f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1879,17 +1879,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v34-0002-Add-pg_depend.refobjversion.patchtext/x-patch; charset=US-ASCII; name=v34-0002-Add-pg_depend.refobjversion.patchDownload
From 432b3ade17660a92516db62aac8529cb9aa1e6cb Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Fri, 23 Oct 2020 13:54:31 +1300
Subject: [PATCH v34 2/3] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. A follow-up commit will use this to record dependencies on
collation versions for indexes, but similar ideas for other kinds of
objects have also been mooted so we want an extensible model.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ++++++
src/backend/catalog/dependency.c | 14 +++++---
src/backend/catalog/pg_depend.c | 14 +++++---
src/bin/initdb/initdb.c | 44 +++++++++++------------
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 +++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 +--
8 files changed, 61 insertions(+), 32 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index d0d2598290..c3f324f05e 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3302,6 +3302,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object.
+ </para>
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..1a927377e7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1600,7 +1600,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1687,7 +1689,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1707,7 +1711,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2679,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 454e569fa9..09c30b13e8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -115,6 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the dependency. Note we don't bother to check for duplicate
* dependencies; there's no harm in them.
*/
+ memset(slot[slot_stored_count]->tts_isnull, false,
+ slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+
slot[slot_stored_count]->tts_values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
@@ -122,9 +127,10 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot[slot_stored_count]->tts_values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
-
- memset(slot[slot_stored_count]->tts_isnull, false,
- slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+ if (version)
+ slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ slot[slot_stored_count]->tts_isnull[Anum_pg_depend_refobjversion - 1] = true;
ExecStoreVirtualTuple(slot[slot_stored_count]);
slot_stored_count++;
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index ee3bfa82f4..daf60d0c77 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1539,55 +1539,55 @@ setup_depend(FILE *cmdfd)
"DELETE FROM pg_shdepend;\n\n",
"VACUUM pg_shdepend;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_class;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_proc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_type;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_cast;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_constraint;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_conversion;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_attrdef;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_language;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_operator;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opclass;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_opfamily;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_am;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amop;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_amproc;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_rewrite;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_trigger;\n\n",
/*
* restriction here to avoid pinning the public namespace
*/
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_namespace "
" WHERE nspname LIKE 'pg%';\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_parser;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_dict;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_template;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_ts_config;\n\n",
- "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p' "
+ "INSERT INTO pg_depend SELECT 0,0,0, tableoid,oid,0, 'p',NULL"
" FROM pg_collation;\n\n",
"INSERT INTO pg_shdepend SELECT 0,0,0,0, tableoid,oid, 'p' "
" FROM pg_authid;\n\n",
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v34-0003-Track-collation-versions-for-indexes.patchtext/x-patch; charset=US-ASCII; name=v34-0003-Track-collation-versions-for-indexes.patchDownload
From 3553c88caf1ca09fe77a1eb35d09b98158e6259f Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Fri, 23 Oct 2020 14:25:30 +1300
Subject: [PATCH v34 3/3] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. Warn that the index may be corrupted
if the versions don't match.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 3 +-
doc/src/sgml/charset.sgml | 35 +++
doc/src/sgml/func.sgml | 4 +-
doc/src/sgml/ref/alter_index.sgml | 17 ++
doc/src/sgml/ref/pgupgrade.sgml | 15 ++
doc/src/sgml/ref/reindex.sgml | 9 +
src/backend/catalog/dependency.c | 211 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 198 +++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 46 +++-
src/backend/catalog/pg_type.c | 60 +++++
src/backend/commands/collationcmds.c | 16 +-
src/backend/commands/tablecmds.c | 31 +++
src/backend/commands/vacuum.c | 32 +++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/optimizer/util/plancat.c | 9 +
src/backend/parser/gram.y | 8 +
src/backend/utils/adt/pg_locale.c | 46 +++-
src/backend/utils/adt/pg_upgrade_support.c | 1 +
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 182 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/pg_dump/t/002_pg_dump.pl | 254 ++++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 1 +
src/bin/psql/tab-complete.c | 27 +-
src/include/catalog/dependency.h | 25 +-
src/include/catalog/index.h | 6 +
src/include/catalog/pg_type.h | 2 +
src/include/nodes/parsenodes.h | 4 +-
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 3 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 58 ++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 211 +++++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 137 ++++++++++
src/tools/pgindent/typedefs.list | 2 +
45 files changed, 1556 insertions(+), 150 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c3f324f05e..c03c8a9611 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3308,7 +3308,8 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<structfield>refobjversion</structfield> <type>text</type>
</para>
<para>
- An optional version for the referenced object.
+ An optional version for the referenced object. Currently used for
+ collations (see <xref linkend="collation-versions"/>).
</para>
</entry>
</row>
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 2745b44417..785d2afe9e 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,41 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The ordering defined by a collation is not necessarily fixed over time.
+ When a collation changes because of an operating system upgrade,
+ persistent data structures such as B-trees that depend on key order might
+ be corrupted.
+ <productname>PostgreSQL</productname> defends against this by recording the
+ current version of each referenced collation for any index that depends on
+ it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes that information available. If the
+ provider later begins to report a different version, a warning will be
+ issued when the index is accessed, until the <xref linkend="sql-reindex"/>
+ or <xref linkend="sql-alterindex"/> command is used to update the version.
+ </para>
+ <para>
+ Version information is available for collations from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems) and Windows.
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 0398b4909b..bf6004f321 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25444,7 +25444,9 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. <literal>null</literal> is returned
+ on operating systems where <productname>PostgreSQL</productname>
+ doesn't have support for versions.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 793119d2fc..8e55b681d5 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,22 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ This command declares that the index is compatible with the currently
+ installed version of a collation that determines its order. It is used
+ to silence warnings caused by collation version incompatibilities and
+ should be issued only if the collation ordering is known not to have
+ changed since the index was last built. Be aware that incorrect use of
+ this command can hide index corruption. If you don't know whether a
+ collation's definition has changed, using <xref linkend="sql-reindex"/>
+ is a safe alternative.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index b59c5697a3..6a23559c5b 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,21 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--index-collation-versions-unknown</option></term>
+ <listitem>
+ <para>
+ When upgrading text indexes from a version of
+ <productname>PostgreSQL</productname> before 14,
+ <application>pg_upgrade</application> assumes by default that the
+ collations used match the current versions (see
+ <xref linkend="collation-versions"/>). Specify
+ <option>--index-collation-versions-unknown</option> to make them as
+ needing to be rebuilt.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index fa43e3a972..7ad5e34894 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -38,6 +38,15 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
several scenarios in which to use <command>REINDEX</command>:
<itemizedlist>
+ <listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index.
+ See <xref linkend="collation-versions"/> for more information.
+ </para>
+ </listitem>
+
<listitem>
<para>
An index has become corrupted, and no longer contains valid
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..c84a3e03d8 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,7 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +436,84 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that 'object' depend on. If the function
+ * returns true, refobjversion will be updated in the catalog.
+ */
+void
+visitDependenciesOf(const ObjectAddress *object,
+ VisitDependenciesOfCB callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ /* Does the callback want to update the version? */
+ if (callback(&otherObject,
+ isnull ? NULL : TextDatumGetCString(depversion),
+ &new_version,
+ userdata))
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ if (new_version)
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1646,38 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Record dependencies on a list of collations, optionally with their current
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ eliminate_duplicate_dependencies(addrs);
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+
+ free_object_addresses(addrs);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1700,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1718,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1746,17 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1812,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1834,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1857,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1896,44 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+
+ /*
+ * Otherwise, it may be a composite type having underlying
+ * collations.
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll))
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1958,9 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant.
*/
- if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(con->constcollid))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,8 +2049,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, param->paramtype, 0,
context->addrs);
/* and its collation, just as for Consts */
- if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(param->paramcollid))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,8 +2136,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, fselect->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(fselect->resultcollid))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,8 +2166,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, relab->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(relab->resultcollid))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,8 +2178,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, iocoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(iocoerce->resultcollid))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,8 +2190,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, acoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(acoerce->resultcollid))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,8 +2278,7 @@ find_expr_references_walker(Node *node,
if (OidIsValid(wc->endInRangeFunc))
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
- if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ if (OidIsValid(wc->inRangeColl))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2423,7 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2445,7 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2841,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..fc5140daa6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2336,7 +2336,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2346,7 +2346,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3665,7 +3665,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..c9ee415a73 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/rel.h"
@@ -1020,6 +1022,8 @@ index_create(Relation heapRelation,
ObjectAddress myself,
referenced;
ObjectAddresses *addrs;
+ List *colls = NIL,
+ *colls_no_version = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1103,30 +1107,65 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* placeholder for normal dependencies */
- addrs = new_object_addresses();
-
- /* Store dependency on collations */
-
- /* The default collation is pinned, so don't bother recording it */
+ /* Get dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- add_exact_object_address(&referenced, addrs);
+ Oid opclass = classObjectId[i];
+
+ /*
+ * The *_pattern_ops opclasses are special: they explicitly do
+ * not depend on collation order so we can save some effort.
+ *
+ * XXX With more analysis, we could also skip version tracking
+ * for some cases like hash indexes with deterministic
+ * collations, because they will never need to order strings.
+ */
+ if (opclass == TEXT_BTREE_PATTERN_OPS_OID ||
+ opclass == VARCHAR_BTREE_PATTERN_OPS_OID ||
+ opclass == BPCHAR_BTREE_PATTERN_OPS_OID)
+ colls_no_version = lappend_oid(colls_no_version, colloid);
+ else
+ colls = lappend_oid(colls, colloid);
+ }
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ /*
+ * Even though there is no top-level collation, there may be
+ * collations affecting ordering inside types, so look there
+ * too.
+ */
+ colls = list_concat(colls, GetTypeCollations(att->atttypid));
}
}
+ /*
+ * If we have anything in both lists, keep just the versioned one to
+ * save some duplication.
+ */
+ if (colls_no_version != NIL && colls != NIL)
+ colls_no_version = list_difference_oid(colls_no_version, colls);
+
+ /* Store the versioned and unversioned collation dependencies. */
+ if (colls_no_version != NIL)
+ recordDependencyOnCollations(&myself, colls_no_version, false);
+ if (colls != NIL)
+ recordDependencyOnCollations(&myself, colls, true);
+
/* Store dependency on operator classes */
+ addrs = new_object_addresses();
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
ObjectAddressSet(referenced, OperatorClassRelationId, classObjectId[i]);
add_exact_object_address(&referenced, addrs);
}
-
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
free_object_addresses(addrs);
@@ -1137,7 +1176,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
/* Store dependencies on anything mentioned in predicate */
@@ -1147,7 +1186,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1226,6 +1265,130 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+typedef struct do_collation_version_check_context
+{
+ Oid relid;
+ List *checked_colls;
+} do_collation_version_check_context;
+
+/*
+ * Raise a warning if the recorded and current collation version don't match.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_collation_version_check(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ do_collation_version_check_context *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations with a version. */
+ if (!version || otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ if (!current_version)
+ return false;
+
+ /*
+ * If we've already checked this collation, skip it. We don't expect too
+ * many duplicates, but it's possible, and we don't want to generate
+ * duplicate warnings. We don't expect their versions to differ.
+ */
+ if (list_member_oid(context->checked_colls, otherObject->objectId))
+ return false;
+
+ /* Do they match? */
+ if (strcmp(current_version, version) != 0)
+ {
+ /*
+ * The version has changed, probably due to an OS/library upgrade or
+ * streaming replication between different OS/library versions.
+ */
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(context->relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+
+ /* Remember not to complain about this collation again. */
+ context->checked_colls = lappend_oid(context->checked_colls,
+ otherObject->objectId);
+
+ return false;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given index.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ do_collation_version_check_context context;
+ ObjectAddress object;
+
+ /*
+ * The callback needs the relid for error messages, and some scratch space
+ * to avoid duplicate warnings.
+ */
+ context.relid = relid;
+ context.checked_colls = NIL;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+
+ visitDependenciesOf(&object, &do_collation_version_check, &context);
+
+ list_free(context.checked_colls);
+}
+
+/*
+ * Update the version for collations. A callback for visitDependenciesOf().
+ */
+static bool
+do_collation_version_update(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ Oid *coll = data;
+
+ /* We only care about dependencies on collations with versions. */
+ if (!version || otherObject->classId != CollationRelationId)
+ return false;
+
+ /* If we're only trying to update one collation, skip others. */
+ if (OidIsValid(*coll) && otherObject->objectId != *coll)
+ return false;
+
+ *new_version = get_collation_version_for_oid(otherObject->objectId);
+
+ return true;
+}
+
+/*
+ * Record the current versions of one or all collations that an index depends
+ * on. InvalidOid means all.
+ */
+void
+index_update_collation_versions(Oid relid, Oid coll)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependenciesOf(&object, &do_collation_version_update, &coll);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1686,6 +1849,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ /* Now we have the old index's collation versions, so fix that. */
+ CommandCounterIncrement();
+ index_update_collation_versions(newIndexId, InvalidOid);
+
/*
* Copy over statistics from old to new index
*/
@@ -3635,6 +3802,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId, InvalidOid);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 0d70cb0c3c..93774c9d21 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -362,7 +362,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 09c30b13e8..01bd17599a 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,6 +28,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
@@ -45,19 +47,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record is added even if the referenced
+ * object is pinned, and the dependency version will be retrieved according to
+ * the referenced object kind. For now, only collation version is
+ * supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -66,6 +73,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
max_slots,
slot_init_count,
slot_stored_count;
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -96,12 +104,38 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* For now we only know how to deal with collations. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX don't need version tracking. */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ version = get_collation_version_for_oid(referenced->objectId);
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (isObjectPinned(referenced, dependDesc))
+ if (!ignore_systempin && isObjectPinned(referenced, dependDesc))
continue;
if (slot_init_count < max_slots)
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 0b04dff773..44eed1a0b3 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,65 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/*
+ * Get a list of all distinct collations that the given type depends on.
+ */
+List *
+GetTypeCollations(Oid typeoid)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result, GetTypeCollations(rangeid));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..7b31272734 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,23 +270,9 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
if (version)
PG_RETURN_TEXT_P(cstring_to_text(version));
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a29c14bf1c..84cda0f24b 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -559,6 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3986,6 +3988,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4160,6 +4166,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4738,6 +4750,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17582,3 +17599,17 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * This overrides an existing dependency on a specific collation for a specific
+ * index to depend on the current collation version.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ index_update_collation_versions(rel->rd_id,
+ get_collation_oid(coll, false));
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 1b6717f727..b1423a0444 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ac8b57109c..530aac68a7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3215,6 +3215,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 4f0da51c26..52c01eb86b 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 60cf7242a3..357ab93fb6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2591,6 +2591,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..3b0324ce18 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,45 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string for a given collation OID.
+ * Return NULL if the provider doesn't support versions.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..8a5953881e 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9224e2ffed..66393becfb 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5934,6 +5935,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 6c79319164..4d49d2b96a 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -179,6 +179,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int coll_unknown;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9851022a53..cb266313a0 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,9 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, bool coll_unknown,
+ Archive *fount);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
{NULL, 0, NULL, 0}
};
@@ -712,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions only relevant in binary upgrade mode */
+ if (dopt.coll_unknown && !dopt.binary_upgrade)
+ fatal("option --index-collation-versions-unknown only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7031,7 +7040,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependcollnames,
+ i_inddependcollversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7067,7 +7078,64 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(quote_ident(ns.nspname) || '.' || quote_ident(c.collname) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend d "
+ " JOIN pg_catalog.pg_collation c ON (c.oid = d.refobjid) "
+ " JOIN pg_catalog.pg_namespace ns ON (c.collnamespace = ns.oid) "
+ " WHERE d.classid = 'pg_catalog.pg_class'::regclass AND "
+ " d.objid = i.indexrelid AND "
+ " d.objsubid = 0 AND "
+ " d.refclassid = 'pg_catalog.pg_collation'::regclass AND "
+ " d.refobjversion IS NOT NULL) AS inddependcollnames, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = 'pg_catalog.pg_class'::regclass AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = 'pg_catalog.pg_collation'::regclass AND "
+ " refobjversion IS NOT NULL) AS inddependcollversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7092,7 +7160,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7131,7 +7201,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7166,7 +7238,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7197,7 +7271,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7231,7 +7307,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7271,6 +7349,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependcollnames = PQfnumber(res, "inddependcollnames");
+ i_inddependcollversions = PQfnumber(res, "inddependcollversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7296,6 +7376,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependcollnames = pg_strdup(PQgetvalue(res, j, i_inddependcollnames));
+ indxinfo[j].inddependcollversions = pg_strdup(PQgetvalue(res, j, i_inddependcollversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16362,7 +16444,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16429,6 +16512,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->coll_unknown, fout);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16457,6 +16544,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->coll_unknown, fout);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18441,6 +18543,70 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Generate calls to binary_update_set_index_coll_version() to import the
+ * collation versions into the new database.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ bool coll_unknown, Archive *fout)
+{
+ char *inddependcollnames = indxinfo->inddependcollnames;
+ char *inddependcollversions = indxinfo->inddependcollversions;
+ char **inddependcollnamesarray;
+ char **inddependcollversionsarray;
+ int ninddependcollnames;
+ int ninddependcollversions;
+
+ /*
+ * By default, a new cluster's index will have pg_depends rows with current
+ * collation versions, meaning that we assume the index isn't corrupted if
+ * importing from a release that didn't record versions. If
+ * --index-collation-versions-unknown was passed in, then we assume such
+ * indexes might be corrupted, and clobber versions with 'unknown' to
+ * trigger version warnings.
+ */
+ if (coll_unknown)
+ {
+ appendPQExpBuffer(buffer,
+ "\n-- For binary upgrade, clobber new index's collation versions\n");
+ appendPQExpBuffer(buffer,
+ "UPDATE pg_catalog.pg_depend SET refobjversion = 'unknown' WHERE objid = '%u'::pg_catalog.oid AND refclassid = 'pg_catalog.pg_collation'::regclass AND refobjversion IS NOT NULL;\n",
+ indxinfo->dobj.catId.oid);
+ }
+
+ /* Restore the versions that were recorded by the old cluster (if any). */
+ parsePGArray(inddependcollnames,
+ &inddependcollnamesarray,
+ &ninddependcollnames);
+ parsePGArray(inddependcollversions,
+ &inddependcollversionsarray,
+ &ninddependcollversions);
+ Assert(ninddependcollnames == ninddependcollversions);
+
+ if (ninddependcollnames > 0)
+ appendPQExpBufferStr(buffer,
+ "\n-- For binary upgrade, restore old index's collation versions\n");
+ for (int i = 0; i < ninddependcollnames; i++)
+ {
+ /*
+ * Import refobjversion from the old cluster, being careful to resolve
+ * the collation OID by name in the new cluster.
+ */
+ appendPQExpBuffer(buffer,
+ "UPDATE pg_catalog.pg_depend SET refobjversion = %s WHERE objid = '%u'::pg_catalog.oid AND refclassid = 'pg_catalog.pg_collation'::regclass AND refobjversion IS NOT NULL AND refobjid = ",
+ inddependcollversionsarray[i],
+ indxinfo->dobj.catId.oid);
+ appendStringLiteralAH(buffer,inddependcollnamesarray[i], fout);
+ appendPQExpBuffer(buffer, "::regcollation;\n");
+ }
+
+ if (inddependcollnamesarray)
+ free(inddependcollnamesarray);
+ if (inddependcollversionsarray)
+ free(inddependcollversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..317bb83970 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,8 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependcollnames; /* FQ names of depended-on collations */
+ char *inddependcollversions; /* versions of the above */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if a partition, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..9dcf18d0e9 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_unknown => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_unknown.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--index-collation-versions-unknown',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_unknown.sql",
+ "$tempdir/binary_coll_unknown.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_unknown => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_unknown => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_unknown => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_unknown => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,59 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ 'index->collation refobjversion' => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore old index's collation versions\E\n
+ \QUPDATE pg_catalog.pg_depend SET refobjversion = 'not_a_version' WHERE objid = '\E\d+\Q'::pg_catalog.oid AND refclassid = 'pg_catalog.pg_collation'::regclass AND refobjversion IS NOT NULL AND refobjid = 'pg_catalog."fr-x-icu"'::regcollation;\E/xm,
+ like => { binary_upgrade => 1 },
+ icu => 1,
+ },
+
+ 'index->collation refobjversion with "unknown" as default' => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_col2(id integer, val text);
+ CREATE INDEX regress_coll_idx2 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx2\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx2 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, clobber new index's collation versions\E\n
+ \QUPDATE pg_catalog.pg_depend SET refobjversion = 'unknown' WHERE objid = '\E\d+\Q'::pg_catalog.oid AND refclassid = 'pg_catalog.pg_collation'::regclass AND refobjversion IS NOT NULL;\E\n
+ \n
+ \Q-- For binary upgrade, restore old index's collation versions\E\n
+ \QUPDATE pg_catalog.pg_depend SET refobjversion = 'not_a_version' WHERE objid = '\E\d+\Q'::pg_catalog.oid AND refclassid = 'pg_catalog.pg_collation'::regclass AND refobjversion IS NOT NULL AND refobjid = 'pg_catalog."fr-x-icu"'::regcollation;\E/xm,
+ like => { binary_coll_unknown => 1 },
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init(extra => [ '--encoding=UTF8' ]);
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init(extra => [ '--encoding=UTF8' ]);
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3411,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3465,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3535,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3594,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..20e73be361 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.ind_coll_unknown ?
+ "--index-collation-versions-unknown" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..548d648e8c 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"index-collation-versions-unknown", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.ind_coll_unknown = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --index-collation-versions-unknown\n"));
+ printf(_(" mark text indexes as needing to be rebuilt\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..398c6d1a97 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,7 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool ind_coll_unknown; /* unknown collation versions corrupted */
} UserOpts;
typedef struct
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b2b4f1fd4d..912eb7409f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -820,6 +821,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1715,7 +1730,8 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
@@ -1765,6 +1781,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev4_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..c4ed5740b6 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -160,7 +160,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +181,30 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef bool(*VisitDependenciesOfCB) (const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data);
+
+extern void visitDependenciesOf(const ObjectAddress *object,
+ VisitDependenciesOfCB callback,
+ void *data);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +223,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..fdffda227d 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +133,10 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid, Oid coll);
+extern void index_import_collation_versions(Oid relid, Oid coll,
+ char *import_version);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 6ae6edf7e0..d228efffc9 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -368,6 +368,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 319f77013f..e1aeea2560 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1853,7 +1853,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1869,6 +1870,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..c5ffea40f2 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check been done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index 9774f534d9..14cde4f5ba 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..ba05f3568b
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,58 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 6;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init(extra => [ '--encoding=UTF8' ]);
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index ebcaeb44fe..c1b5d4aabd 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -769,10 +769,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..da5a03663d 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,217 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+-----------------------------------+------------+---------------------
+ icuidx00_val | "fr-x-icu" | up to date
+ icuidx00_val_val | "fr-x-icu" | up to date
+ icuidx00_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val | "fr-x-icu" | up to date
+ icuidx00_val_val_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_where | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_where | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(63 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..6cb7786e13 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,14 +2065,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2092,14 +2094,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c9afb88f7d 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,143 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
-- cleanup
RESET search_path;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b6acade6c6..03c4e0fe5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2914,6 +2914,8 @@ dlist_head
dlist_iter
dlist_mutable_iter
dlist_node
+do_collation_version_check_context
+do_collation_version_update_context
ds_state
dsa_area
dsa_area_control
--
2.20.1
Re: Collation versioning
On Sat, Oct 31, 2020 at 10:28 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Fri, Oct 30, 2020 at 1:20 AM Thomas Munro <thomas.munro@gmail.com> wrote:
4. I didn't really like the use of '' for unknown. I figured out how
to use NULL for that.Hrmph. That logic didn't work for the pattern ops case. I fixed
that, by partially reintroducing a special value, but this time
restricting the code that knows about that to just pg_dump, and I used
the value 'unknown', so really it's not special at all as far as the
server is concerned and there is only one kind of warning message.
Ok, I'm fine with that.
Furthermore, I realised that I really don't like the policy of
assuming that all text-related indexes imported from older releases
need the "unknown" warning. That'll just make this feature
unnecessarily noisy and unpopular when 14 comes out, essentially
crying wolf, even though it's technically true that the collations in
imported-from-before-14 indexes are of unknown version. Even worse,
instructions might be shared around the internet to show how to shut
the warnings up without reindexing, and then later when there really
is a version change, someone might find those instructions and follow
them! So I propose that the default should be to assume that indexes
are not corrupted, unless you opt into the more pessimistic policy
with --index-collation-versions-unknown. Done like that.
Yes, I was also worried about spamming this kind of messages after an
upgrade. Note that this was initially planned for REL_13_STABLE,
whose release date was very close to glibc 2.28, so at that time this
would have been more likely to have a real corruption on the indexes.
I'm fine with the new behavior.
I also realised that I don't like carrying a bunch of C code to
support binary upgrades, when it's really just a hand-coded trivial
UPDATE of pg_depend. Is there any reason pg_dump --binary-upgrade
shouldn't just dump UPDATE statements, and make this whole feature a
lot less mysterious, and shorter? Done like that.
I just thought that it wouldn't be acceptable to do plain DML on the
catalogs. If that's ok, then definitely this approach is better.
While testing on another OS that will be encountered in the build farm
when I commit this, I realised that I needed to add --encoding=UTF8 to
tests under src/bin/pg_dump and src/test/locale, because they now do
things with ICU collations (if built with ICU support) and that only
works with UTF8.
Oh I didn't know that.
Another option would be to find a way to skip those
tests if the encoding is not UTF8. Hmm, I wonder if it's bad to
effectively remove the testing that comes for free from buildfarm
animals running this under non-UTF8 encodings; but if we actually
valued that, I suppose we'd do it explicitly as another test pass with
SQL_ASCII.
I guess it would be better to keep checking non-UTF8 encodings, but
the current approach looks quite random and I don't have any better
suggestions.
Note that v34 now fails when run on a without that don't have
defined(__GLIBC__) (e.g. macos). The failure are in
collate.icu.utf8.sql, of the form:
- icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "default" | version not trackedGiven the new approach, the only option I can see is to simply remove
any attempt to cover the default collation in the tests. An
alternative file would be a pain to maintain and it wouldn't bring any
value apart from checking that the default collation is either always
tracked or never, but not a mix of those.
Re: Collation versioning
On Mon, Nov 2, 2020 at 7:59 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
Note that v34 now fails when run on a without that don't have
defined(__GLIBC__) (e.g. macos). The failure are in
collate.icu.utf8.sql, of the form:- icuidx06_d_en_fr_ga | "default" | up to date + icuidx06_d_en_fr_ga | "default" | version not trackedGiven the new approach, the only option I can see is to simply remove
any attempt to cover the default collation in the tests. An
alternative file would be a pain to maintain and it wouldn't bring any
value apart from checking that the default collation is either always
tracked or never, but not a mix of those.
Blah, right. Ok, I changed it to output XXX for "default".
I did a bit more language clean up. I fixed the tab completion for
ALTER INDEX ... ALTER COLLATION. I simplified a couple of tests. I
dropped the pg_dump TAP test for now (I might see if I can find a
simpler way to test refobjversion restoration later). I dropped the
special handling for T_CollateExpr in find_expr_references_walker()
(it wasn't affecting the test cases we have, and I wasn't entirely
sure about it; do you see a problem?). I dropped the VACUUM-log-spam
feature for now (I'm not against it, but it seems like an independent
enough thing to not include in the initial commit). This brought the
patch mercifully under 100kB. This is the version I'm planning to
commit if you don't see anything else.
Attachments:
v35-0001-Remove-pg_collation.collversion.patchtext/x-patch; charset=US-ASCII; name=v35-0001-Remove-pg_collation.collversion.patchDownload
From 6d66b2233f5e8a7e61500c9e9427bd69c3c3a4b5 Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Mon, 2 Nov 2020 19:36:09 +1300
Subject: [PATCH v35 1/3] Remove pg_collation.collversion.
This model couldn't be extended to cover the default collation, and
didn't have any information about the affected database objects when the
version changed. Remove, in preparation for a follow-up commit that
will add a new mechanism.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 ---
doc/src/sgml/func.sgml | 6 +-
doc/src/sgml/ref/alter_collation.sgml | 63 -------------
doc/src/sgml/ref/create_collation.sgml | 21 -----
src/backend/catalog/pg_collation.c | 5 --
src/backend/commands/collationcmds.c | 88 -------------------
src/backend/nodes/copyfuncs.c | 13 ---
src/backend/nodes/equalfuncs.c | 11 ---
src/backend/parser/gram.y | 18 +---
src/backend/tcop/utility.c | 12 ---
src/backend/utils/adt/pg_locale.c | 37 --------
src/bin/pg_dump/pg_dump.c | 24 +----
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_collation.dat | 7 +-
src/include/catalog/pg_collation.h | 5 --
src/include/catalog/toasting.h | 1 -
src/include/commands/collationcmds.h | 1 -
src/include/nodes/parsenodes.h | 11 ---
.../regress/expected/collate.icu.utf8.out | 3 -
.../regress/expected/collate.linux.utf8.out | 3 -
src/test/regress/sql/collate.icu.utf8.sql | 5 --
src/test/regress/sql/collate.linux.utf8.sql | 5 --
22 files changed, 8 insertions(+), 344 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 0ccdff1cda..d0d2598290 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -2361,17 +2361,6 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<symbol>LC_CTYPE</symbol> for this collation object
</para></entry>
</row>
-
- <row>
- <entry role="catalog_table_entry"><para role="column_definition">
- <structfield>collversion</structfield> <type>text</type>
- </para>
- <para>
- Provider-specific version of the collation. This is recorded when the
- collation is created and then checked when it is used, to detect
- changes in the collation definition that could lead to data corruption.
- </para></entry>
- </row>
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d8eee3a826..0398b4909b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25444,11 +25444,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system. If this is different from the
- value in
- <structname>pg_collation</structname>.<structfield>collversion</structfield>,
- then objects depending on the collation might need to be rebuilt. See
- also <xref linkend="sql-altercollation"/>.
+ installed in the operating system.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_collation.sgml b/doc/src/sgml/ref/alter_collation.sgml
index af9ff2867b..65429aabe2 100644
--- a/doc/src/sgml/ref/alter_collation.sgml
+++ b/doc/src/sgml/ref/alter_collation.sgml
@@ -21,8 +21,6 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
-ALTER COLLATION <replaceable>name</replaceable> REFRESH VERSION
-
ALTER COLLATION <replaceable>name</replaceable> RENAME TO <replaceable>new_name</replaceable>
ALTER COLLATION <replaceable>name</replaceable> OWNER TO { <replaceable>new_owner</replaceable> | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_schema</replaceable>
@@ -88,70 +86,9 @@ ALTER COLLATION <replaceable>name</replaceable> SET SCHEMA <replaceable>new_sche
</listitem>
</varlistentry>
- <varlistentry>
- <term><literal>REFRESH VERSION</literal></term>
- <listitem>
- <para>
- Update the collation's version.
- See <xref linkend="sql-altercollation-notes"/> below.
- </para>
- </listitem>
- </varlistentry>
</variablelist>
</refsect1>
- <refsect1 id="sql-altercollation-notes" xreflabel="Notes">
- <title>Notes</title>
-
- <para>
- When using collations provided by the ICU library, the ICU-specific version
- of the collator is recorded in the system catalog when the collation object
- is created. When the collation is used, the current version is
- checked against the recorded version, and a warning is issued when there is
- a mismatch, for example:
-<screen>
-WARNING: collation "xx-x-icu" has version mismatch
-DETAIL: The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
-HINT: Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
-</screen>
- A change in collation definitions can lead to corrupt indexes and other
- problems because the database system relies on stored objects having a
- certain sort order. Generally, this should be avoided, but it can happen
- in legitimate circumstances, such as when
- using <command>pg_upgrade</command> to upgrade to server binaries linked
- with a newer version of ICU. When this happens, all objects depending on
- the collation should be rebuilt, for example,
- using <command>REINDEX</command>. When that is done, the collation version
- can be refreshed using the command <literal>ALTER COLLATION ... REFRESH
- VERSION</literal>. This will update the system catalog to record the
- current collator version and will make the warning go away. Note that this
- does not actually check whether all affected objects have been rebuilt
- correctly.
- </para>
- <para>
- When using collations provided by <literal>libc</literal> and
- <productname>PostgreSQL</productname> was built with the GNU C library, the
- C library's version is used as a collation version. Since collation
- definitions typically change only with GNU C library releases, this provides
- some defense against corruption, but it is not completely reliable.
- </para>
- <para>
- Currently, there is no version tracking for the database default collation.
- </para>
-
- <para>
- The following query can be used to identify all collations in the current
- database that need to be refreshed and the objects that depend on them:
-<programlisting><![CDATA[
-SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
- pg_describe_object(classid, objid, objsubid) AS "Object"
- FROM pg_depend d JOIN pg_collation c
- ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
- WHERE c.collversion <> pg_collation_actual_version(c.oid)
- ORDER BY 1, 2;
-]]></programlisting></para>
- </refsect1>
-
<refsect1>
<title>Examples</title>
diff --git a/doc/src/sgml/ref/create_collation.sgml b/doc/src/sgml/ref/create_collation.sgml
index 58f5f0cd63..b97842071f 100644
--- a/doc/src/sgml/ref/create_collation.sgml
+++ b/doc/src/sgml/ref/create_collation.sgml
@@ -27,7 +27,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> (
[ LC_CTYPE = <replaceable>lc_ctype</replaceable>, ]
[ PROVIDER = <replaceable>provider</replaceable>, ]
[ DETERMINISTIC = <replaceable>boolean</replaceable>, ]
- [ VERSION = <replaceable>version</replaceable> ]
)
CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replaceable>existing_collation</replaceable>
</synopsis>
@@ -149,26 +148,6 @@ CREATE COLLATION [ IF NOT EXISTS ] <replaceable>name</replaceable> FROM <replace
</listitem>
</varlistentry>
- <varlistentry>
- <term><replaceable>version</replaceable></term>
-
- <listitem>
- <para>
- Specifies the version string to store with the collation. Normally,
- this should be omitted, which will cause the version to be computed
- from the actual version of the collation as provided by the operating
- system. This option is intended to be used
- by <command>pg_upgrade</command> for copying the version from an
- existing installation.
- </para>
-
- <para>
- See also <xref linkend="sql-altercollation"/> for how to handle
- collation version mismatches.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry>
<term><replaceable>existing_collation</replaceable></term>
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 5fdf1acb7e..3c84378d02 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -49,7 +49,6 @@ CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet)
{
@@ -167,10 +166,6 @@ CollationCreate(const char *collname, Oid collnamespace,
values[Anum_pg_collation_collcollate - 1] = NameGetDatum(&name_collate);
namestrcpy(&name_ctype, collctype);
values[Anum_pg_collation_collctype - 1] = NameGetDatum(&name_ctype);
- if (collversion)
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(collversion);
- else
- nulls[Anum_pg_collation_collversion - 1] = true;
tup = heap_form_tuple(tupDesc, values, nulls);
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 9f6582c530..5ad8886e60 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -61,14 +61,12 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
DefElem *lcctypeEl = NULL;
DefElem *providerEl = NULL;
DefElem *deterministicEl = NULL;
- DefElem *versionEl = NULL;
char *collcollate = NULL;
char *collctype = NULL;
char *collproviderstr = NULL;
bool collisdeterministic = true;
int collencoding = 0;
char collprovider = 0;
- char *collversion = NULL;
Oid newoid;
ObjectAddress address;
@@ -96,8 +94,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
defelp = &providerEl;
else if (strcmp(defel->defname, "deterministic") == 0)
defelp = &deterministicEl;
- else if (strcmp(defel->defname, "version") == 0)
- defelp = &versionEl;
else
{
ereport(ERROR,
@@ -166,9 +162,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
if (deterministicEl)
collisdeterministic = defGetBoolean(deterministicEl);
- if (versionEl)
- collversion = defGetString(versionEl);
-
if (collproviderstr)
{
if (pg_strcasecmp(collproviderstr, "icu") == 0)
@@ -215,9 +208,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
}
}
- if (!collversion)
- collversion = get_collation_actual_version(collprovider, collcollate);
-
newoid = CollationCreate(collName,
collNamespace,
GetUserId(),
@@ -226,7 +216,6 @@ DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_e
collencoding,
collcollate,
collctype,
- collversion,
if_not_exists,
false); /* not quiet */
@@ -277,80 +266,6 @@ IsThereCollationInNamespace(const char *collname, Oid nspOid)
collname, get_namespace_name(nspOid))));
}
-/*
- * ALTER COLLATION
- */
-ObjectAddress
-AlterCollation(AlterCollationStmt *stmt)
-{
- Relation rel;
- Oid collOid;
- HeapTuple tup;
- Form_pg_collation collForm;
- Datum collversion;
- bool isnull;
- char *oldversion;
- char *newversion;
- ObjectAddress address;
-
- rel = table_open(CollationRelationId, RowExclusiveLock);
- collOid = get_collation_oid(stmt->collname, false);
-
- if (!pg_collation_ownercheck(collOid, GetUserId()))
- aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_COLLATION,
- NameListToString(stmt->collname));
-
- tup = SearchSysCacheCopy1(COLLOID, ObjectIdGetDatum(collOid));
- if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for collation %u", collOid);
-
- collForm = (Form_pg_collation) GETSTRUCT(tup);
- collversion = SysCacheGetAttr(COLLOID, tup, Anum_pg_collation_collversion,
- &isnull);
- oldversion = isnull ? NULL : TextDatumGetCString(collversion);
-
- newversion = get_collation_actual_version(collForm->collprovider, NameStr(collForm->collcollate));
-
- /* cannot change from NULL to non-NULL or vice versa */
- if ((!oldversion && newversion) || (oldversion && !newversion))
- elog(ERROR, "invalid collation version change");
- else if (oldversion && newversion && strcmp(newversion, oldversion) != 0)
- {
- bool nulls[Natts_pg_collation];
- bool replaces[Natts_pg_collation];
- Datum values[Natts_pg_collation];
-
- ereport(NOTICE,
- (errmsg("changing version from %s to %s",
- oldversion, newversion)));
-
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replaces, false, sizeof(replaces));
-
- values[Anum_pg_collation_collversion - 1] = CStringGetTextDatum(newversion);
- replaces[Anum_pg_collation_collversion - 1] = true;
-
- tup = heap_modify_tuple(tup, RelationGetDescr(rel),
- values, nulls, replaces);
- }
- else
- ereport(NOTICE,
- (errmsg("version has not changed")));
-
- CatalogTupleUpdate(rel, &tup->t_self, tup);
-
- InvokeObjectPostAlterHook(CollationRelationId, collOid, 0);
-
- ObjectAddressSet(address, CollationRelationId, collOid);
-
- heap_freetuple(tup);
- table_close(rel, NoLock);
-
- return address;
-}
-
-
Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
@@ -608,7 +523,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(localebuf, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
localebuf, localebuf,
- get_collation_actual_version(COLLPROVIDER_LIBC, localebuf),
true, true);
if (OidIsValid(collid))
{
@@ -669,7 +583,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
collid = CollationCreate(alias, nspid, GetUserId(),
COLLPROVIDER_LIBC, true, enc,
locale, locale,
- get_collation_actual_version(COLLPROVIDER_LIBC, locale),
true, true);
if (OidIsValid(collid))
{
@@ -731,7 +644,6 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
nspid, GetUserId(),
COLLPROVIDER_ICU, true, -1,
collcollate, collcollate,
- get_collation_actual_version(COLLPROVIDER_ICU, collcollate),
true, true);
if (OidIsValid(collid))
{
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..ac8b57109c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3224,16 +3224,6 @@ _copyAlterTableCmd(const AlterTableCmd *from)
return newnode;
}
-static AlterCollationStmt *
-_copyAlterCollationStmt(const AlterCollationStmt *from)
-{
- AlterCollationStmt *newnode = makeNode(AlterCollationStmt);
-
- COPY_NODE_FIELD(collname);
-
- return newnode;
-}
-
static AlterDomainStmt *
_copyAlterDomainStmt(const AlterDomainStmt *from)
{
@@ -5229,9 +5219,6 @@ copyObjectImpl(const void *from)
case T_AlterTableCmd:
retval = _copyAlterTableCmd(from);
break;
- case T_AlterCollationStmt:
- retval = _copyAlterCollationStmt(from);
- break;
case T_AlterDomainStmt:
retval = _copyAlterDomainStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..0cf90ef33c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1107,14 +1107,6 @@ _equalAlterTableCmd(const AlterTableCmd *a, const AlterTableCmd *b)
return true;
}
-static bool
-_equalAlterCollationStmt(const AlterCollationStmt *a, const AlterCollationStmt *b)
-{
- COMPARE_NODE_FIELD(collname);
-
- return true;
-}
-
static bool
_equalAlterDomainStmt(const AlterDomainStmt *a, const AlterDomainStmt *b)
{
@@ -3283,9 +3275,6 @@ equal(const void *a, const void *b)
case T_AlterTableCmd:
retval = _equalAlterTableCmd(a, b);
break;
- case T_AlterCollationStmt:
- retval = _equalAlterCollationStmt(a, b);
- break;
case T_AlterDomainStmt:
retval = _equalAlterDomainStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..60cf7242a3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -254,7 +254,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
}
%type <node> stmt schema_stmt
- AlterEventTrigStmt AlterCollationStmt
+ AlterEventTrigStmt
AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt
AlterFdwStmt AlterForeignServerStmt AlterGroupStmt
AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt
@@ -835,7 +835,6 @@ stmtmulti: stmtmulti ';' stmt
stmt :
AlterEventTrigStmt
- | AlterCollationStmt
| AlterDatabaseStmt
| AlterDatabaseSetStmt
| AlterDefaultPrivilegesStmt
@@ -10169,21 +10168,6 @@ drop_option:
}
;
-/*****************************************************************************
- *
- * ALTER COLLATION
- *
- *****************************************************************************/
-
-AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P
- {
- AlterCollationStmt *n = makeNode(AlterCollationStmt);
- n->collname = $3;
- $$ = (Node *)n;
- }
- ;
-
-
/*****************************************************************************
*
* ALTER SYSTEM
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..f398027fa6 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1842,10 +1842,6 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterStatistics((AlterStatsStmt *) parsetree);
break;
- case T_AlterCollationStmt:
- address = AlterCollation((AlterCollationStmt *) parsetree);
- break;
-
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -2993,10 +2989,6 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_DROP_SUBSCRIPTION;
break;
- case T_AlterCollationStmt:
- tag = CMDTAG_ALTER_COLLATION;
- break;
-
case T_PrepareStmt:
tag = CMDTAG_PREPARE;
break;
@@ -3609,10 +3601,6 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
- case T_AlterCollationStmt:
- lev = LOGSTMT_DDL;
- break;
-
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 07299dbc09..514e0fa0af 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1513,8 +1513,6 @@ pg_newlocale_from_collation(Oid collid)
const char *collctype pg_attribute_unused();
struct pg_locale_struct result;
pg_locale_t resultp;
- Datum collversion;
- bool isnull;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1616,41 +1614,6 @@ pg_newlocale_from_collation(Oid collid)
#endif /* not USE_ICU */
}
- collversion = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
- {
- char *actual_versionstr;
- char *collversionstr;
-
- actual_versionstr = get_collation_actual_version(collform->collprovider, collcollate);
- if (!actual_versionstr)
- {
- /*
- * This could happen when specifying a version in CREATE
- * COLLATION for a libc locale, or manually creating a mess in
- * the catalogs.
- */
- ereport(ERROR,
- (errmsg("collation \"%s\" has no actual version, but a version was specified",
- NameStr(collform->collname))));
- }
- collversionstr = TextDatumGetCString(collversion);
-
- if (strcmp(actual_versionstr, collversionstr) != 0)
- ereport(WARNING,
- (errmsg("collation \"%s\" has version mismatch",
- NameStr(collform->collname)),
- errdetail("The collation in the database was created using version %s, "
- "but the operating system provides version %s.",
- collversionstr, actual_versionstr),
- errhint("Rebuild all objects affected by this collation and run "
- "ALTER COLLATION %s REFRESH VERSION, "
- "or build PostgreSQL with the right library version.",
- quote_qualified_identifier(get_namespace_name(collform->collnamespace),
- NameStr(collform->collname)))));
- }
-
ReleaseSysCache(tp);
/* We'll keep the pg_locale_t structures in TopMemoryContext */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 03f9d4d9e8..9851022a53 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13589,12 +13589,10 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
if (fout->remoteVersion >= 100000)
appendPQExpBufferStr(query,
- "collprovider, "
- "collversion, ");
+ "collprovider, ");
else
appendPQExpBufferStr(query,
- "'c' AS collprovider, "
- "NULL AS collversion, ");
+ "'c' AS collprovider, ");
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(query,
@@ -13655,24 +13653,6 @@ dumpCollation(Archive *fout, CollInfo *collinfo)
appendStringLiteralAH(q, collctype, fout);
}
- /*
- * For binary upgrade, carry over the collation version. For normal
- * dump/restore, omit the version, so that it is computed upon restore.
- */
- if (dopt->binary_upgrade)
- {
- int i_collversion;
-
- i_collversion = PQfnumber(res, "collversion");
- if (!PQgetisnull(res, 0, i_collversion))
- {
- appendPQExpBufferStr(q, ", version = ");
- appendStringLiteralAH(q,
- PQgetvalue(res, 0, i_collversion),
- fout);
- }
- }
-
appendPQExpBufferStr(q, ");\n");
if (dopt->binary_upgrade)
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index 73650f88e9..fbb729a0b2 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -53,6 +53,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202010291
+#define CATALOG_VERSION_NO 202011011
#endif
diff --git a/src/include/catalog/pg_collation.dat b/src/include/catalog/pg_collation.dat
index ba1b3e201b..45301ccdd7 100644
--- a/src/include/catalog/pg_collation.dat
+++ b/src/include/catalog/pg_collation.dat
@@ -15,17 +15,16 @@
{ oid => '100', oid_symbol => 'DEFAULT_COLLATION_OID',
descr => 'database\'s default collation',
collname => 'default', collnamespace => 'PGNSP', collowner => 'PGUID',
- collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '',
- collversion => '_null_' },
+ collprovider => 'd', collencoding => '-1', collcollate => '', collctype => '' },
{ oid => '950', oid_symbol => 'C_COLLATION_OID',
descr => 'standard C collation',
collname => 'C', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'C',
- collctype => 'C', collversion => '_null_' },
+ collctype => 'C' },
{ oid => '951', oid_symbol => 'POSIX_COLLATION_OID',
descr => 'standard POSIX collation',
collname => 'POSIX', collnamespace => 'PGNSP', collowner => 'PGUID',
collprovider => 'c', collencoding => '-1', collcollate => 'POSIX',
- collctype => 'POSIX', collversion => '_null_' },
+ collctype => 'POSIX' },
]
diff --git a/src/include/catalog/pg_collation.h b/src/include/catalog/pg_collation.h
index 27618b324d..e7e958b808 100644
--- a/src/include/catalog/pg_collation.h
+++ b/src/include/catalog/pg_collation.h
@@ -37,10 +37,6 @@ CATALOG(pg_collation,3456,CollationRelationId)
int32 collencoding; /* encoding for this collation; -1 = "all" */
NameData collcollate; /* LC_COLLATE setting */
NameData collctype; /* LC_CTYPE setting */
-#ifdef CATALOG_VARLEN /* variable-length fields start here */
- text collversion; /* provider-dependent version of collation
- * data */
-#endif
} FormData_pg_collation;
/* ----------------
@@ -65,7 +61,6 @@ extern Oid CollationCreate(const char *collname, Oid collnamespace,
bool collisdeterministic,
int32 collencoding,
const char *collcollate, const char *collctype,
- const char *collversion,
bool if_not_exists,
bool quiet);
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8f131893dc 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -51,7 +51,6 @@ extern void BootstrapToastTable(char *relName,
/* normal catalogs */
DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
-DECLARE_TOAST(pg_collation, 4161, 4162);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
DECLARE_TOAST(pg_description, 2834, 2835);
diff --git a/src/include/commands/collationcmds.h b/src/include/commands/collationcmds.h
index 373b85374c..3e1c16ac7f 100644
--- a/src/include/commands/collationcmds.h
+++ b/src/include/commands/collationcmds.h
@@ -20,6 +20,5 @@
extern ObjectAddress DefineCollation(ParseState *pstate, List *names, List *parameters, bool if_not_exists);
extern void IsThereCollationInNamespace(const char *collname, Oid nspOid);
-extern ObjectAddress AlterCollation(AlterCollationStmt *stmt);
#endif /* COLLATIONCMDS_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ff584f2955..319f77013f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1879,17 +1879,6 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
} AlterTableCmd;
-/* ----------------------
- * Alter Collation
- * ----------------------
- */
-typedef struct AlterCollationStmt
-{
- NodeTag type;
- List *collname;
-} AlterCollationStmt;
-
-
/* ----------------------
* Alter Domain
*
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 2b86ce9028..60d9263a2f 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1082,9 +1082,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/expected/collate.linux.utf8.out b/src/test/regress/expected/collate.linux.utf8.out
index f06ae543e4..580b00eea7 100644
--- a/src/test/regress/expected/collate.linux.utf8.out
+++ b/src/test/regress/expected/collate.linux.utf8.out
@@ -1093,9 +1093,6 @@ SELECT collname FROM pg_collation WHERE collname LIKE 'test%';
DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-ALTER COLLATION "en_US" REFRESH VERSION;
-NOTICE: version has not changed
-- dependencies
CREATE COLLATION test0 FROM "C";
CREATE TABLE collate_dep_test1 (a int, b text COLLATE test0);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 67de7d9794..35acf91fbf 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -405,11 +405,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en-x-icu" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
diff --git a/src/test/regress/sql/collate.linux.utf8.sql b/src/test/regress/sql/collate.linux.utf8.sql
index cbbd2203e4..c697c99488 100644
--- a/src/test/regress/sql/collate.linux.utf8.sql
+++ b/src/test/regress/sql/collate.linux.utf8.sql
@@ -406,11 +406,6 @@ DROP SCHEMA test_schema;
DROP ROLE regress_test_role;
--- ALTER
-
-ALTER COLLATION "en_US" REFRESH VERSION;
-
-
-- dependencies
CREATE COLLATION test0 FROM "C";
--
2.20.1
v35-0002-Add-pg_depend.refobjversion.patchtext/x-patch; charset=US-ASCII; name=v35-0002-Add-pg_depend.refobjversion.patchDownload
From 14b72deac6ceed07f91239b5d68bcc58ec8f6e8c Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Mon, 2 Nov 2020 19:40:49 +1300
Subject: [PATCH v35 2/3] Add pg_depend.refobjversion.
Provide a place for the version of referenced database objects to be
recorded. A follow-up commit will use this to record dependencies on
collation versions for indexes, but similar ideas for other kinds of
objects have also been mooted.
Author: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 11 +++++++++++
src/backend/catalog/dependency.c | 14 ++++++++++----
src/backend/catalog/pg_depend.c | 14 ++++++++++----
src/include/catalog/catversion.h | 2 +-
src/include/catalog/dependency.h | 1 +
src/include/catalog/pg_depend.h | 4 ++++
src/include/catalog/toasting.h | 1 +
src/test/regress/expected/misc_sanity.out | 4 ++--
8 files changed, 40 insertions(+), 11 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index d0d2598290..c3f324f05e 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3302,6 +3302,17 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
A code defining the specific semantics of this dependency relationship; see text
</para></entry>
</row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>refobjversion</structfield> <type>text</type>
+ </para>
+ <para>
+ An optional version for the referenced object.
+ </para>
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..1a927377e7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1600,7 +1600,9 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -1687,7 +1689,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, self_addrs->numrefs,
+ self_addrs->refs,
+ self_addrs->numrefs,
+ NULL,
self_behavior);
else
{
@@ -1707,7 +1711,9 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, context.addrs->numrefs,
+ context.addrs->refs,
+ context.addrs->numrefs,
+ NULL,
behavior);
free_object_addresses(context.addrs);
@@ -2679,7 +2685,7 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs,
+ referenced->refs, referenced->numrefs, NULL,
behavior);
}
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 454e569fa9..09c30b13e8 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -24,6 +24,7 @@
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
@@ -44,7 +45,7 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
}
/*
@@ -55,6 +56,7 @@ void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior)
{
Relation dependDesc;
@@ -115,6 +117,9 @@ recordMultipleDependencies(const ObjectAddress *depender,
* Record the dependency. Note we don't bother to check for duplicate
* dependencies; there's no harm in them.
*/
+ memset(slot[slot_stored_count]->tts_isnull, false,
+ slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+
slot[slot_stored_count]->tts_values[Anum_pg_depend_refclassid - 1] = ObjectIdGetDatum(referenced->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjid - 1] = ObjectIdGetDatum(referenced->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjsubid - 1] = Int32GetDatum(referenced->objectSubId);
@@ -122,9 +127,10 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot[slot_stored_count]->tts_values[Anum_pg_depend_classid - 1] = ObjectIdGetDatum(depender->classId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objid - 1] = ObjectIdGetDatum(depender->objectId);
slot[slot_stored_count]->tts_values[Anum_pg_depend_objsubid - 1] = Int32GetDatum(depender->objectSubId);
-
- memset(slot[slot_stored_count]->tts_isnull, false,
- slot[slot_stored_count]->tts_tupleDescriptor->natts * sizeof(bool));
+ if (version)
+ slot[slot_stored_count]->tts_values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ slot[slot_stored_count]->tts_isnull[Anum_pg_depend_refobjversion - 1] = true;
ExecStoreVirtualTuple(slot[slot_stored_count]);
slot_stored_count++;
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index fbb729a0b2..6610e3c23f 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -53,6 +53,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202011011
+#define CATALOG_VERSION_NO 202011012
#endif
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3baa5e498a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -189,6 +189,7 @@ extern void recordDependencyOn(const ObjectAddress *depender,
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
+ const char *version,
DependencyType behavior);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index ccf0a98330..7489022795 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -61,6 +61,10 @@ CATALOG(pg_depend,2608,DependRelationId)
* field. See DependencyType in catalog/dependency.h.
*/
char deptype; /* see codes in dependency.h */
+#ifdef CATALOG_VARLEN
+ text refobjversion; /* version tracking, NULL if not used or
+ * unknown */
+#endif
} FormData_pg_depend;
/* ----------------
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 8f131893dc..e320d82203 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -53,6 +53,7 @@ DECLARE_TOAST(pg_aggregate, 4159, 4160);
DECLARE_TOAST(pg_attrdef, 2830, 2831);
DECLARE_TOAST(pg_constraint, 2832, 2833);
DECLARE_TOAST(pg_default_acl, 4143, 4144);
+DECLARE_TOAST(pg_depend, 8888, 8889);
DECLARE_TOAST(pg_description, 2834, 2835);
DECLARE_TOAST(pg_event_trigger, 4145, 4146);
DECLARE_TOAST(pg_extension, 4147, 4148);
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..d40afeef78 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -18,8 +18,8 @@ WHERE refclassid = 0 OR refobjid = 0 OR
deptype NOT IN ('a', 'e', 'i', 'n', 'p') OR
(deptype != 'p' AND (classid = 0 OR objid = 0)) OR
(deptype = 'p' AND (classid != 0 OR objid != 0 OR objsubid != 0));
- classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
----------+-------+----------+------------+----------+-------------+---------
+ classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype | refobjversion
+---------+-------+----------+------------+----------+-------------+---------+---------------
(0 rows)
-- **************** pg_shdepend ****************
--
2.20.1
v35-0003-Track-collation-versions-for-indexes.patchtext/x-patch; charset=US-ASCII; name=v35-0003-Track-collation-versions-for-indexes.patchDownload
From 8dbd7c945c2088000287f922a9c8b9dfeca52363 Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Mon, 2 Nov 2020 19:50:45 +1300
Subject: [PATCH v35 3/3] Track collation versions for indexes.
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. When accessing the index later, warn
that the index may be corrupted if the current version doesn't match.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Reviewed-by: Laurenz Albe <laurenz.albe@cybertec.at>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/catalogs.sgml | 3 +-
doc/src/sgml/charset.sgml | 38 ++++
doc/src/sgml/func.sgml | 4 +-
doc/src/sgml/ref/alter_index.sgml | 15 ++
doc/src/sgml/ref/pgupgrade.sgml | 15 ++
doc/src/sgml/ref/reindex.sgml | 9 +
src/backend/catalog/dependency.c | 182 +++++++++++++---
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 198 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 46 +++-
src/backend/catalog/pg_type.c | 60 ++++++
src/backend/commands/collationcmds.c | 16 +-
src/backend/commands/tablecmds.c | 31 +++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/optimizer/util/plancat.c | 9 +
src/backend/parser/gram.y | 8 +
src/backend/utils/adt/pg_locale.c | 46 +++-
src/backend/utils/adt/pg_upgrade_support.c | 1 +
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 182 +++++++++++++++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 1 +
src/bin/psql/tab-complete.c | 29 ++-
src/include/catalog/catversion.h | 2 +-
src/include/catalog/dependency.h | 25 ++-
src/include/catalog/index.h | 3 +
src/include/catalog/pg_depend.h | 3 +-
src/include/catalog/pg_type.h | 2 +
src/include/nodes/parsenodes.h | 4 +-
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 3 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 67 ++++++
.../regress/expected/collate.icu.utf8.out | 201 ++++++++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 132 ++++++++++++
src/tools/pgindent/typedefs.list | 2 +
43 files changed, 1288 insertions(+), 94 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c3f324f05e..5fb9dca425 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -3308,7 +3308,8 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<structfield>refobjversion</structfield> <type>text</type>
</para>
<para>
- An optional version for the referenced object.
+ An optional version for the referenced object. Currently used for
+ indexes' collations (see <xref linkend="collation-versions"/>).
</para>
</entry>
</row>
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 2745b44417..832a701523 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -948,6 +948,44 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
</tip>
</sect3>
</sect2>
+
+ <sect2 id="collation-versions">
+ <title>Collation Versions</title>
+
+ <para>
+ The sort order defined by a collation is not necessarily fixed over time.
+ <productname>PostgreSQL</productname> relies on external libraries that
+ are subject to operating system upgrades, and can also differ between
+ servers involved in binary replication and file-system-level migration.
+ Persistent data structures such as B-trees that depend on sort order might
+ be corrupted by any resulting change.
+ <productname>PostgreSQL</productname> defends against this by recording the
+ current version of each referenced collation for any index that depends on
+ it in the
+ <link linkend="catalog-pg-depend"><structname>pg_depend</structname></link>
+ catalog, if the collation provider makes that information available. If the
+ provider later begins to report a different version, a warning will be
+ issued when the index is accessed, until either the
+ <xref linkend="sql-reindex"/> command or the
+ <xref linkend="sql-alterindex"/> command is used to update the version.
+ </para>
+ <para>
+ Version information is available from the
+ <literal>icu</literal> provider on all operating systems. For the
+ <literal>libc</literal> provider, versions are currently only available
+ on systems using the GNU C library (most Linux systems) and Windows.
+ </para>
+
+ <note>
+ <para>
+ When using the GNU C library for collations, the C library's version
+ is used as a proxy for the collation version. Many Linux distributions
+ change collation definitions only when upgrading the C library, but this
+ approach is imperfect as maintainers are free to back-port newer
+ collation definitions to older C library releases.
+ </para>
+ </note>
+ </sect2>
</sect1>
<sect1 id="multibyte">
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 0398b4909b..bf6004f321 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25444,7 +25444,9 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. <literal>null</literal> is returned
+ on operating systems where <productname>PostgreSQL</productname>
+ doesn't have support for versions.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 793119d2fc..214005a86c 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,7 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_number</replaceable>
@@ -112,6 +113,20 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
+ <listitem>
+ <para>
+ Silences warnings about mismatched collation versions, by declaring
+ that the index is compatible with the current collation definition.
+ Be aware that incorrect use of this command can hide index corruption.
+ If you don't know whether a collation's definition has changed
+ incompatibly, <xref linkend="sql-reindex"/> is a safe alternative.
+ See <xref linkend="collation-versions"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index b59c5697a3..dbbfface5c 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,21 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--index-collation-versions-unknown</option></term>
+ <listitem>
+ <para>
+ When upgrading indexes from a version of
+ <productname>PostgreSQL</productname> before 14,
+ <application>pg_upgrade</application> assumes by default that any
+ collations used when building the index match the current versions (see
+ <xref linkend="collation-versions"/>). Specify
+ <option>--index-collation-versions-unknown</option> to mark them as
+ needing to be rebuilt instead.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index fa43e3a972..7ad5e34894 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -38,6 +38,15 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
several scenarios in which to use <command>REINDEX</command>:
<itemizedlist>
+ <listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index.
+ See <xref linkend="collation-versions"/> for more information.
+ </para>
+ </listitem>
+
<listitem>
<para>
An index has become corrupted, and no longer contains valid
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..b0d037600e 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -434,6 +435,84 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that 'object' depend on. If the function
+ * returns true, refobjversion will be updated in the catalog.
+ */
+void
+visitDependenciesOf(const ObjectAddress *object,
+ VisitDependenciesOfCB callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ /* Does the callback want to update the version? */
+ if (callback(&otherObject,
+ isnull ? NULL : TextDatumGetCString(depversion),
+ &new_version,
+ userdata))
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ if (new_version)
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,38 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Record dependencies on a list of collations, optionally with their current
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ eliminate_duplicate_dependencies(addrs);
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+
+ free_object_addresses(addrs);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1602,8 +1713,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,7 +1741,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool record_version)
{
find_expr_references_context context;
RangeTblEntry rte;
@@ -1691,8 +1803,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ record_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1825,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ record_version);
free_object_addresses(context.addrs);
}
@@ -1770,6 +1882,29 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Top-level collation if valid */
+ if (OidIsValid(var->varcollid))
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ /* Otherwise, it may be a type with internal collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll))
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
}
/*
@@ -1794,11 +1929,9 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant.
*/
- if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(con->constcollid))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,8 +2020,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, param->paramtype, 0,
context->addrs);
/* and its collation, just as for Consts */
- if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(param->paramcollid))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,8 +2107,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, fselect->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(fselect->resultcollid))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,8 +2137,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, relab->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(relab->resultcollid))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,8 +2149,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, iocoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(iocoerce->resultcollid))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,8 +2161,7 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_TYPE, acoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
- if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(acoerce->resultcollid))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,8 +2249,7 @@ find_expr_references_walker(Node *node,
if (OidIsValid(wc->endInRangeFunc))
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
- if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ if (OidIsValid(wc->inRangeColl))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2394,7 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2416,7 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2812,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9ccf378d45..4cd7d76938 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2336,7 +2336,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2346,7 +2346,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3706,7 +3706,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index a18cc7cad3..227053218c 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/rel.h"
@@ -1020,6 +1022,8 @@ index_create(Relation heapRelation,
ObjectAddress myself,
referenced;
ObjectAddresses *addrs;
+ List *colls = NIL,
+ *colls_no_version = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1103,30 +1107,65 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* placeholder for normal dependencies */
- addrs = new_object_addresses();
-
- /* Store dependency on collations */
-
- /* The default collation is pinned, so don't bother recording it */
+ /* Get dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- add_exact_object_address(&referenced, addrs);
+ Oid opclass = classObjectId[i];
+
+ /*
+ * The *_pattern_ops opclasses are special: they explicitly do
+ * not depend on collation order so we can save some effort.
+ *
+ * XXX With more analysis, we could also skip version tracking
+ * for some cases like hash indexes with deterministic
+ * collations, because they will never need to order strings.
+ */
+ if (opclass == TEXT_BTREE_PATTERN_OPS_OID ||
+ opclass == VARCHAR_BTREE_PATTERN_OPS_OID ||
+ opclass == BPCHAR_BTREE_PATTERN_OPS_OID)
+ colls_no_version = lappend_oid(colls_no_version, colloid);
+ else
+ colls = lappend_oid(colls, colloid);
+ }
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ /*
+ * Even though there is no top-level collation, there may be
+ * collations affecting ordering inside types, so look there
+ * too.
+ */
+ colls = list_concat(colls, GetTypeCollations(att->atttypid));
}
}
+ /*
+ * If we have anything in both lists, keep just the versioned one to
+ * avoid some duplication.
+ */
+ if (colls_no_version != NIL && colls != NIL)
+ colls_no_version = list_difference_oid(colls_no_version, colls);
+
+ /* Store the versioned and unversioned collation dependencies. */
+ if (colls_no_version != NIL)
+ recordDependencyOnCollations(&myself, colls_no_version, false);
+ if (colls != NIL)
+ recordDependencyOnCollations(&myself, colls, true);
+
/* Store dependency on operator classes */
+ addrs = new_object_addresses();
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
ObjectAddressSet(referenced, OperatorClassRelationId, classObjectId[i]);
add_exact_object_address(&referenced, addrs);
}
-
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
free_object_addresses(addrs);
@@ -1137,7 +1176,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
/* Store dependencies on anything mentioned in predicate */
@@ -1147,7 +1186,7 @@ index_create(Relation heapRelation,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1226,6 +1265,130 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+typedef struct do_collation_version_check_context
+{
+ Oid relid;
+ List *checked_colls;
+} do_collation_version_check_context;
+
+/*
+ * Raise a warning if the recorded and current collation version don't match.
+ * This is a callback for visitDependenciesOf().
+ */
+static bool
+do_collation_version_check(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ do_collation_version_check_context *context = data;
+ char *current_version;
+
+ /* We only care about dependencies on collations with a version. */
+ if (!version || otherObject->classId != CollationRelationId)
+ return false;
+
+ /* Ask the provider for the current version. Give up if unsupported. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ if (!current_version)
+ return false;
+
+ /*
+ * If we've already checked this collation, skip it. We don't expect too
+ * many duplicates, but it's possible, and we don't want to generate
+ * duplicate warnings. We don't expect their versions to differ.
+ */
+ if (list_member_oid(context->checked_colls, otherObject->objectId))
+ return false;
+
+ /* Do they match? */
+ if (strcmp(current_version, version) != 0)
+ {
+ /*
+ * The version has changed, probably due to an OS/library upgrade or
+ * streaming replication between different OS/library versions.
+ */
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(context->relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+
+ /* Remember not to complain about this collation again. */
+ context->checked_colls = lappend_oid(context->checked_colls,
+ otherObject->objectId);
+
+ return false;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given index.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ do_collation_version_check_context context;
+ ObjectAddress object;
+
+ /*
+ * The callback needs the relid for error messages, and some scratch space
+ * to avoid duplicate warnings.
+ */
+ context.relid = relid;
+ context.checked_colls = NIL;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+
+ visitDependenciesOf(&object, &do_collation_version_check, &context);
+
+ list_free(context.checked_colls);
+}
+
+/*
+ * Update the version for collations. A callback for visitDependenciesOf().
+ */
+static bool
+do_collation_version_update(const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data)
+{
+ Oid *coll = data;
+
+ /* We only care about dependencies on collations with versions. */
+ if (!version || otherObject->classId != CollationRelationId)
+ return false;
+
+ /* If we're only trying to update one collation, skip others. */
+ if (OidIsValid(*coll) && otherObject->objectId != *coll)
+ return false;
+
+ *new_version = get_collation_version_for_oid(otherObject->objectId);
+
+ return true;
+}
+
+/*
+ * Record the current versions of one or all collations that an index depends
+ * on. InvalidOid means all.
+ */
+void
+index_update_collation_versions(Oid relid, Oid coll)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependenciesOf(&object, &do_collation_version_update, &coll);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1686,6 +1849,10 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ /* Now we have the old index's collation versions, so fix that. */
+ CommandCounterIncrement();
+ index_update_collation_versions(newIndexId, InvalidOid);
+
/*
* Copy over statistics from old to new index
*/
@@ -3638,6 +3805,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId, InvalidOid);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 0d70cb0c3c..93774c9d21 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -362,7 +362,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 09c30b13e8..25290c821f 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,6 +28,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
@@ -45,19 +47,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If record_version is true, then a record is added even if the referenced
+ * object is pinned, and the dependency version will be retrieved according to
+ * the referenced object kind. For now, only collation version is
+ * supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool record_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -66,6 +73,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
max_slots,
slot_init_count,
slot_stored_count;
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -96,12 +104,38 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (record_version)
+ {
+ /* For now we only know how to deal with collations. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX don't need version tracking. */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ version = get_collation_version_for_oid(referenced->objectId);
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (isObjectPinned(referenced, dependDesc))
+ if (!ignore_systempin && isObjectPinned(referenced, dependDesc))
continue;
if (slot_init_count < max_slots)
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 0b04dff773..44eed1a0b3 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,65 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/*
+ * Get a list of all distinct collations that the given type depends on.
+ */
+List *
+GetTypeCollations(Oid typeoid)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result, GetTypeCollations(rangeid));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..7b31272734 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,23 +270,9 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
if (version)
PG_RETURN_TEXT_P(cstring_to_text(version));
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index df13b72974..1b105ba1c4 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -93,6 +93,7 @@
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/partcache.h"
+#include "utils/pg_locale.h"
#include "utils/relcache.h"
#include "utils/ruleutils.h"
#include "utils/snapmgr.h"
@@ -559,6 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
+static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
/* ----------------------------------------------------------------
@@ -3986,6 +3988,10 @@ AlterTableGetLockLevel(List *cmds)
cmd_lockmode = AccessShareLock;
break;
+ case AT_AlterCollationRefreshVersion:
+ cmd_lockmode = AccessExclusiveLock;
+ break;
+
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4160,6 +4166,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* This command never recurses */
pass = AT_PASS_MISC;
break;
+ case AT_AlterCollationRefreshVersion: /* ALTER COLLATION ... REFRESH
+ * VERSION */
+ ATSimplePermissions(rel, ATT_INDEX);
+ /* This command never recurses */
+ pass = AT_PASS_MISC;
+ break;
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_FOREIGN_TABLE);
ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
@@ -4738,6 +4750,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterCollationRefreshVersion:
+ /* ATPrepCmd ensured it must be an index */
+ Assert(rel->rd_rel->relkind == RELKIND_INDEX);
+ ATExecAlterCollationRefreshVersion(rel, cmd->object);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -17582,3 +17599,17 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * ALTER INDEX ... ALTER COLLATION ... REFRESH VERSION
+ *
+ * Update refobjversion to the current collation version by force. This clears
+ * warnings about version mismatches without the need to run REINDEX,
+ * potentially hiding corruption due to ordering changes.
+ */
+static void
+ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
+{
+ index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
+ CacheInvalidateRelcache(rel);
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ac8b57109c..530aac68a7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3215,6 +3215,7 @@ _copyAlterTableCmd(const AlterTableCmd *from)
COPY_SCALAR_FIELD(subtype);
COPY_STRING_FIELD(name);
+ COPY_NODE_FIELD(object);
COPY_SCALAR_FIELD(num);
COPY_NODE_FIELD(newowner);
COPY_NODE_FIELD(def);
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 4f0da51c26..52c01eb86b 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 60cf7242a3..357ab93fb6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2591,6 +2591,14 @@ alter_table_cmd:
n->subtype = AT_NoForceRowSecurity;
$$ = (Node *)n;
}
+ /* ALTER INDEX <name> ALTER COLLATION ... REFRESH VERSION */
+ | ALTER COLLATION any_name REFRESH VERSION_P
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterCollationRefreshVersion;
+ n->object = $3;
+ $$ = (Node *)n;
+ }
| alter_generic_options
{
AlterTableCmd *n = makeNode(AlterTableCmd);
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..3b0324ce18 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,45 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string for a given collation OID.
+ * Return NULL if the provider doesn't support versions.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..8a5953881e 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9224e2ffed..66393becfb 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5934,6 +5935,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 6c79319164..4d49d2b96a 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -179,6 +179,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int coll_unknown;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9851022a53..cf89a7ec6d 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,9 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, bool coll_unknown,
+ Archive *fount);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -385,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"index-collation-versions-unknown", no_argument, &dopt.coll_unknown, 1},
{NULL, 0, NULL, 0}
};
@@ -712,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions only relevant in binary upgrade mode */
+ if (dopt.coll_unknown && !dopt.binary_upgrade)
+ fatal("option --index-collation-versions-unknown only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7031,7 +7040,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependcollnames,
+ i_inddependcollversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7067,7 +7078,64 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(quote_ident(ns.nspname) || '.' || quote_ident(c.collname) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend d "
+ " JOIN pg_catalog.pg_collation c ON (c.oid = d.refobjid) "
+ " JOIN pg_catalog.pg_namespace ns ON (c.collnamespace = ns.oid) "
+ " WHERE d.classid = 'pg_catalog.pg_class'::regclass AND "
+ " d.objid = i.indexrelid AND "
+ " d.objsubid = 0 AND "
+ " d.refclassid = 'pg_catalog.pg_collation'::regclass AND "
+ " d.refobjversion IS NOT NULL) AS inddependcollnames, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = 'pg_catalog.pg_class'::regclass AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = 'pg_catalog.pg_collation'::regclass AND "
+ " refobjversion IS NOT NULL) AS inddependcollversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7092,7 +7160,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7131,7 +7201,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7166,7 +7238,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7197,7 +7271,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7231,7 +7307,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "'{}' AS inddependcollnames, "
+ "'{}' AS inddependcollversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7271,6 +7349,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependcollnames = PQfnumber(res, "inddependcollnames");
+ i_inddependcollversions = PQfnumber(res, "inddependcollversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7296,6 +7376,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependcollnames = pg_strdup(PQgetvalue(res, j, i_inddependcollnames));
+ indxinfo[j].inddependcollversions = pg_strdup(PQgetvalue(res, j, i_inddependcollversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16362,7 +16444,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16429,6 +16512,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->coll_unknown, fout);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16457,6 +16544,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->coll_unknown, fout);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18441,6 +18543,70 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Generate UPDATE statements to import the collation versions into the new
+ * cluster, during a binary upgrade.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ bool coll_unknown, Archive *fout)
+{
+ char *inddependcollnames = indxinfo->inddependcollnames;
+ char *inddependcollversions = indxinfo->inddependcollversions;
+ char **inddependcollnamesarray;
+ char **inddependcollversionsarray;
+ int ninddependcollnames;
+ int ninddependcollversions;
+
+ /*
+ * By default, the new cluster's index will have pg_depends rows with
+ * current collation versions, meaning that we assume the index isn't
+ * corrupted if importing from a release that didn't record versions.
+ * However, if --index-collation-versions-unknown was passed in, then we
+ * assume such indexes might be corrupted, and clobber versions with
+ * 'unknown' to trigger version warnings.
+ */
+ if (coll_unknown)
+ {
+ appendPQExpBuffer(buffer,
+ "\n-- For binary upgrade, clobber new index's collation versions\n");
+ appendPQExpBuffer(buffer,
+ "UPDATE pg_catalog.pg_depend SET refobjversion = 'unknown' WHERE objid = '%u'::pg_catalog.oid AND refclassid = 'pg_catalog.pg_collation'::regclass AND refobjversion IS NOT NULL;\n",
+ indxinfo->dobj.catId.oid);
+ }
+
+ /* Restore the versions that were recorded by the old cluster (if any). */
+ parsePGArray(inddependcollnames,
+ &inddependcollnamesarray,
+ &ninddependcollnames);
+ parsePGArray(inddependcollversions,
+ &inddependcollversionsarray,
+ &ninddependcollversions);
+ Assert(ninddependcollnames == ninddependcollversions);
+
+ if (ninddependcollnames > 0)
+ appendPQExpBufferStr(buffer,
+ "\n-- For binary upgrade, restore old index's collation versions\n");
+ for (int i = 0; i < ninddependcollnames; i++)
+ {
+ /*
+ * Import refobjversion from the old cluster, being careful to resolve
+ * the collation OID by name in the new cluster.
+ */
+ appendPQExpBuffer(buffer,
+ "UPDATE pg_catalog.pg_depend SET refobjversion = %s WHERE objid = '%u'::pg_catalog.oid AND refclassid = 'pg_catalog.pg_collation'::regclass AND refobjversion IS NOT NULL AND refobjid = ",
+ inddependcollversionsarray[i],
+ indxinfo->dobj.catId.oid);
+ appendStringLiteralAH(buffer,inddependcollnamesarray[i], fout);
+ appendPQExpBuffer(buffer, "::regcollation;\n");
+ }
+
+ if (inddependcollnamesarray)
+ free(inddependcollnamesarray);
+ if (inddependcollversionsarray)
+ free(inddependcollversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..317bb83970 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,8 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependcollnames; /* FQ names of depended-on collations */
+ char *inddependcollversions; /* versions of the above */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if a partition, parent index OID */
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..20e73be361 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.ind_coll_unknown ?
+ "--index-collation-versions-unknown" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..548d648e8c 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"index-collation-versions-unknown", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.ind_coll_unknown = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --index-collation-versions-unknown\n"));
+ printf(_(" mark text indexes as needing to be rebuilt\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..19c64513b0 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,7 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool ind_coll_unknown; /* mark unknown index collation versions */
} UserOpts;
typedef struct
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b2b4f1fd4d..5238a960f7 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -45,6 +45,7 @@
#include "catalog/pg_am_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "common.h"
#include "libpq-fe.h"
#include "pqexpbuffer.h"
@@ -820,6 +821,20 @@ static const SchemaQuery Query_for_list_of_statistics = {
" (SELECT tgrelid FROM pg_catalog.pg_trigger "\
" WHERE pg_catalog.quote_ident(tgname)='%s')"
+/* the silly-looking length condition is just to eat up the current word */
+#define Query_for_list_of_colls_for_one_index \
+" SELECT DISTINCT pg_catalog.quote_ident(coll.collname) " \
+" FROM pg_catalog.pg_depend d, pg_catalog.pg_collation coll, " \
+" pg_catalog.pg_class c" \
+" WHERE (%d = pg_catalog.length('%s'))" \
+" AND d.refclassid = " CppAsString2(CollationRelationId) \
+" AND d.refobjid = coll.oid " \
+" AND d.classid = " CppAsString2(RelationRelationId) \
+" AND d.objid = c.oid " \
+" AND c.relkind = " CppAsString2(RELKIND_INDEX) \
+" AND pg_catalog.pg_table_is_visible(c.oid) " \
+" AND c.relname = '%s'"
+
#define Query_for_list_of_ts_configurations \
"SELECT pg_catalog.quote_ident(cfgname) FROM pg_catalog.pg_ts_config "\
" WHERE substring(pg_catalog.quote_ident(cfgname),1,%d)='%s'"
@@ -1715,14 +1730,15 @@ psql_completion(const char *text, int start, int end)
/* ALTER INDEX <name> */
else if (Matches("ALTER", "INDEX", MatchAny))
COMPLETE_WITH("ALTER COLUMN", "OWNER TO", "RENAME TO", "SET",
- "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS");
+ "RESET", "ATTACH PARTITION", "DEPENDS", "NO DEPENDS",
+ "ALTER COLLATION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH"))
COMPLETE_WITH("PARTITION");
else if (Matches("ALTER", "INDEX", MatchAny, "ATTACH", "PARTITION"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_indexes, NULL);
/* ALTER INDEX <name> ALTER */
else if (Matches("ALTER", "INDEX", MatchAny, "ALTER"))
- COMPLETE_WITH("COLUMN");
+ COMPLETE_WITH("COLLATION", "COLUMN");
/* ALTER INDEX <name> ALTER COLUMN */
else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLUMN"))
{
@@ -1765,6 +1781,15 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH("ON EXTENSION");
else if (Matches("ALTER", "INDEX", MatchAny, "DEPENDS"))
COMPLETE_WITH("ON EXTENSION");
+ /* ALTER INDEX <name> ALTER COLLATION */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION"))
+ {
+ completion_info_charp = prev3_wd;
+ COMPLETE_WITH_QUERY(Query_for_list_of_colls_for_one_index);
+ }
+ /* ALTER INDEX <name> ALTER COLLATION <name> */
+ else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLLATION", MatchAny))
+ COMPLETE_WITH("REFRESH VERSION");
/* ALTER LANGUAGE <name> */
else if (Matches("ALTER", "LANGUAGE", MatchAny))
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index 6610e3c23f..f28f083aca 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -53,6 +53,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202011012
+#define CATALOG_VERSION_NO 202011013
#endif
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..901d5019cd 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -160,7 +160,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool record_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +181,30 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef bool(*VisitDependenciesOfCB) (const ObjectAddress *otherObject,
+ const char *version,
+ char **new_version,
+ void *data);
+
+extern void visitDependenciesOf(const ObjectAddress *object,
+ VisitDependenciesOfCB callback,
+ void *data);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool record_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +223,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..f4559b09d7 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,9 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+extern void index_update_collation_versions(Oid relid, Oid coll);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_depend.h b/src/include/catalog/pg_depend.h
index 7489022795..eeafbbe8d7 100644
--- a/src/include/catalog/pg_depend.h
+++ b/src/include/catalog/pg_depend.h
@@ -62,8 +62,7 @@ CATALOG(pg_depend,2608,DependRelationId)
*/
char deptype; /* see codes in dependency.h */
#ifdef CATALOG_VARLEN
- text refobjversion; /* version tracking, NULL if not used or
- * unknown */
+ text refobjversion; /* version of referenced object */
#endif
} FormData_pg_depend;
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 6ae6edf7e0..d228efffc9 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -368,6 +368,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 319f77013f..e1aeea2560 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1853,7 +1853,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1869,6 +1870,7 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
+ List *object; /* collation to act on if it's a collation */
int16 num; /* attribute number for columns referenced by
* number */
RoleSpec *newowner;
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..c5ffea40f2 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check been done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index 9774f534d9..14cde4f5ba 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..a67f78cb71
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,67 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 10;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init(extra => [ '--encoding=UTF8' ]);
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+ my ($ret, $out, $err) = $node->psql('postgres', "SELECT * FROM icu1");
+ is($ret, 0, 'SELECT should succeed.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres', 'CREATE TABLE icu1(val text);');
+$node->safe_psql('postgres', 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+$node->safe_psql('postgres', 'ALTER INDEX icu1_fr ALTER COLLATION "fr-x-icu" REFRESH VERSION;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+$node->safe_psql('postgres', 'REINDEX TABLE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..16b4d9e2cd 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,207 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- two rows expected, only one a version, because we don't try to merge these yet
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- two rows expected with version, because we don't try to merge these yet
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- two rows expected with version (expression walker + attribute)
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+-- two rows expected, one with a version (expression walker + attribute)
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+SELECT objid::regclass::text collate "C", refobjid::regcollation::text collate "C",
+CASE
+WHEN refobjid = 'default'::regcollation THEN 'XXX' -- depends on libc version support
+WHEN refobjversion IS NULL THEN 'version not tracked'
+WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ELSE 'out of date'
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2, 3;
+ objid | refobjid | version
+-----------------------------------+------------+---------------------
+ icuidx00_val | "fr-x-icu" | up to date
+ icuidx00_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val | "fr-x-icu" | up to date
+ icuidx00_val_pattern_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_where | "fr-x-icu" | up to date
+ icuidx00_val_pattern_where | "fr-x-icu" | version not tracked
+ icuidx00_val_val | "fr-x-icu" | up to date
+ icuidx00_val_val_pattern | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | XXX
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | XXX
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | XXX
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | XXX
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | XXX
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | XXX
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+(58 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+---------------
+ icuidx17_part
+(1 row)
+
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
+ objid | ver
+---------------+-----
+ icuidx17_part | f
+(1 row)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 012c1eb067..17f1383ea4 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,14 +2065,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2092,14 +2094,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..4714c044d5 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,138 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- two rows expected, only one a version, because we don't try to merge these yet
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- two rows expected with version, because we don't try to merge these yet
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- two rows expected with version (expression walker + attribute)
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+-- two rows expected, one with a version (expression walker + attribute)
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+
+SELECT objid::regclass::text collate "C", refobjid::regcollation::text collate "C",
+CASE
+WHEN refobjid = 'default'::regcollation THEN 'XXX' -- depends on libc version support
+WHEN refobjversion IS NULL THEN 'version not tracked'
+WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ELSE 'out of date'
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2, 3;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Test ALTER INDEX name ALTER COLLATION name REFRESH VERSION
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part'
+AND refobjversion IS NOT NULL;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ALTER INDEX icuidx17_part ALTER COLLATION "en-x-icu" REFRESH VERSION;
+SELECT objid::regclass, refobjversion = 'not a version' AS ver FROM pg_depend
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text = 'icuidx17_part';
-- cleanup
RESET search_path;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b6acade6c6..03c4e0fe5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2914,6 +2914,8 @@ dlist_head
dlist_iter
dlist_mutable_iter
dlist_node
+do_collation_version_check_context
+do_collation_version_update_context
ds_state
dsa_area
dsa_area_control
--
2.20.1
Re: Collation versioning
On Mon, Nov 2, 2020 at 3:56 PM Thomas Munro <thomas.munro@gmail.com> wrote:
On Mon, Nov 2, 2020 at 7:59 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
Note that v34 now fails when run on a without that don't have
defined(__GLIBC__) (e.g. macos). The failure are in
collate.icu.utf8.sql, of the form:- icuidx06_d_en_fr_ga | "default" | up to date + icuidx06_d_en_fr_ga | "default" | version not trackedGiven the new approach, the only option I can see is to simply remove
any attempt to cover the default collation in the tests. An
alternative file would be a pain to maintain and it wouldn't bring any
value apart from checking that the default collation is either always
tracked or never, but not a mix of those.Blah, right. Ok, I changed it to output XXX for "default".
I did a bit more language clean up. I fixed the tab completion for
ALTER INDEX ... ALTER COLLATION. I simplified a couple of tests. I
dropped the pg_dump TAP test for now (I might see if I can find a
simpler way to test refobjversion restoration later). I dropped the
special handling for T_CollateExpr in find_expr_references_walker()
(it wasn't affecting the test cases we have, and I wasn't entirely
sure about it; do you see a problem?). I dropped the VACUUM-log-spam
feature for now (I'm not against it, but it seems like an independent
enough thing to not include in the initial commit). This brought the
patch mercifully under 100kB. This is the version I'm planning to
commit if you don't see anything else.
Thanks a lot for the updated patch! I agree with dropping the
T_CollateExpr test in find_exp_references_walker(). This was more a
hack than anything, and it should be better addressed by an approach
that can actually handle all cases rather than some random ones.
I'm also ok with dropping the logging from VACUUM from the initial
patch, but this seems like an important codepath to handle (and an
easy one), so I hope we can address that afterwards.
I just have a minor nit:
+ /* Do they match? */
+ if (strcmp(current_version, version) != 0)
+ {
+ /*
+ * The version has changed, probably due to an OS/library upgrade or
+ * streaming replication between different OS/library versions.
+ */
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\"
version \"%s\", but the current version is \"%s\"",
+ get_rel_name(context->relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes
in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+
+ /* Remember not to complain about this collation again. */
+ context->checked_colls = lappend_oid(context->checked_colls,
+ otherObject->objectId);It's maybe not immediately obvious that it's safe to save the
collation oid at that point, or that it'll always be. Maybe move it
in the if branch above to make it extra clear?
and also this will probably need an update:
-#define CATALOG_VERSION_NO 202010291
+#define CATALOG_VERSION_NO 202011013Re: Collation versioning
On Mon, Nov 2, 2020 at 10:28 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
+ /* Remember not to complain about this collation again. */ + context->checked_colls = lappend_oid(context->checked_colls, + otherObject->objectId);It's maybe not immediately obvious that it's safe to save the
collation oid at that point, or that it'll always be. Maybe move it
in the if branch above to make it extra clear?
Ok, moved and renamed, and finally pushed. Thanks for all the help!
Re: Collation versioning
On Mon, Nov 2, 2020 at 9:04 PM Thomas Munro <thomas.munro@gmail.com> wrote:
On Mon, Nov 2, 2020 at 10:28 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
+ /* Remember not to complain about this collation again. */ + context->checked_colls = lappend_oid(context->checked_colls, + otherObject->objectId);It's maybe not immediately obvious that it's safe to save the
collation oid at that point, or that it'll always be. Maybe move it
in the if branch above to make it extra clear?Ok, moved and renamed, and finally pushed. Thanks for all the help!
\o/ Thanks a lot Thomas!
Re: Collation versioning
On Tue, Nov 3, 2020 at 2:08 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
On Mon, Nov 2, 2020 at 9:04 PM Thomas Munro <thomas.munro@gmail.com> wrote:
Ok, moved and renamed, and finally pushed. Thanks for all the help!
\o/ Thanks a lot Thomas!
Hmm, a failure from dory (WIndows) during pg_upgrade:
performing post-bootstrap initialization ... 2020-11-02 08:08:22.213
EST [5392] FATAL: could not get collation version for locale
"English_United States.1252": error code 87
87 means invalid parameter. I'm surprised it got through various
other tests and then failed here. Whelk (also Windows) passed using
"German_Germany.1252". Hmm. I'll wait for more Windows systems to
report.
Re: Collation versioning
Thomas Munro <thomas.munro@gmail.com> writes:
Hmm, a failure from dory (WIndows) during pg_upgrade:
performing post-bootstrap initialization ... 2020-11-02 08:08:22.213
EST [5392] FATAL: could not get collation version for locale
"English_United States.1252": error code 87
87 means invalid parameter. I'm surprised it got through various
other tests and then failed here. Whelk (also Windows) passed using
"German_Germany.1252". Hmm. I'll wait for more Windows systems to
report.
drongo just did it too, and it seems repeatable on dory. I'm not 100%
sure, but I think the buildfarm's initial "check" step may be run under C
locale while pg_upgrade sees whatever the machine's prevailing locale is.
If that's correct, it seems like the simplest explanation is just that
extraction of a collation version is busted for (some?) non-C locales on
Windows. Could be something as dumb as spaces in the locale name
being problematic.
regards, tom lane
Re: Collation versioning
On Tue, Nov 3, 2020 at 6:51 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Thomas Munro <thomas.munro@gmail.com> writes:
Hmm, a failure from dory (WIndows) during pg_upgrade:
performing post-bootstrap initialization ... 2020-11-02 08:08:22.213
EST [5392] FATAL: could not get collation version for locale
"English_United States.1252": error code 8787 means invalid parameter. I'm surprised it got through various
other tests and then failed here. Whelk (also Windows) passed using
"German_Germany.1252". Hmm. I'll wait for more Windows systems to
report.drongo just did it too, and it seems repeatable on dory. I'm not 100%
sure, but I think the buildfarm's initial "check" step may be run under C
locale while pg_upgrade sees whatever the machine's prevailing locale is.
If that's correct, it seems like the simplest explanation is just that
extraction of a collation version is busted for (some?) non-C locales on
Windows. Could be something as dumb as spaces in the locale name
being problematic.
Fortunately David Rowley is able to repro this on his Windows box (it
fails even with strings that are succeeding on the other BF machines),
so we have something to work with. The name mangling that is done in
get_iso_localename() looks pretty interesting... It does feel a bit
like there is some other hidden environmental factor or setting here,
because commit 352f6f2df60 tested OK on Juan Jose's machine too.
Hopefully more soon.
Re: Collation versioning
On Tue, 3 Nov 2020 at 09:43, Thomas Munro <thomas.munro@gmail.com> wrote:
Fortunately David Rowley is able to repro this on his Windows box (it
fails even with strings that are succeeding on the other BF machines),
so we have something to work with. The name mangling that is done in
get_iso_localename() looks pretty interesting... It does feel a bit
like there is some other hidden environmental factor or setting here,
because commit 352f6f2df60 tested OK on Juan Jose's machine too.
Hopefully more soon.
It seems to boil down to GetNLSVersionEx() not liking the "English_New
Zealand.1252" string. The theory about it having a space does not
seem to be a factor as if I change it to "English_Australia.1252", I
get the same issue.
Going by the docs in [1]https://docs.microsoft.com/en-us/windows/win32/api/winnls/nf-winnls-getnlsversionex and following the "local name" link to [2]https://docs.microsoft.com/en-us/windows/win32/intl/locale-names,
there's a description there that mentions: "Generally, the pattern
<language>-<REGION> is used.". So, if I just hack the code in
get_collation_actual_version() to pass "en-NZ" to GetNLSVersionEx(),
that works fine.
In [3]/messages/by-id/CAC+AXB0Eat3aLeTrbDoBB9jX863CU_+RSbgiAjcED5DcXoBoFQ@mail.gmail.com, Juan José was passing in en-US rather than these more weird
Windows-specific locale strings, so the testing that code got when it
went in didn't include seeing if something like "English_New
Zealand.1252" would be accepted.
The "English_New Zealand.1252" string seems to come from the
setlocales() call in initdb via check_locale_name(LC_COLLATE,
lc_collate, &canonname), and fundamentally setlocale(LC_COLLATE).
I'm still a bit mystified why whelk seems unphased by this change. You
can see from [4]https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=whelk&dt=2020-11-02%2020%3A41%3A40&stg=check-pg_upgrade that it must be passing "German_Germany.1252" to
GetNLSVersionEx(). I've tested both on Windows 8.1 and Windows 10 and
I can't get GetNLSVersionEx() to accept that. So maybe Windows 7
allowed these non-ISO formats? That theory seems to break down a bit
when you see that walleye is perfectly happy on Windows 10 (MinGW64).
You can see from [5]https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=walleye&dt=2020-11-02%2020%3A55%3A31&stg=check-pg_upgrade it mentions "The database cluster will be
initialized with locale "English_United States.1252".".
Running low on ideas for now, so thought I'd post this in case it
someone thinks of something else.
David
[1]: https://docs.microsoft.com/en-us/windows/win32/api/winnls/nf-winnls-getnlsversionex
[2]: https://docs.microsoft.com/en-us/windows/win32/intl/locale-names
[3]: /messages/by-id/CAC+AXB0Eat3aLeTrbDoBB9jX863CU_+RSbgiAjcED5DcXoBoFQ@mail.gmail.com
[4]: https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=whelk&dt=2020-11-02%2020%3A41%3A40&stg=check-pg_upgrade
[5]: https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=walleye&dt=2020-11-02%2020%3A55%3A31&stg=check-pg_upgrade
Re: Collation versioning
On Tue, 3 Nov 2020 at 12:29, David Rowley <dgrowleyml@gmail.com> wrote:
Running low on ideas for now, so thought I'd post this in case it
someone thinks of something else.
FWIW, the attached does fix the issue for me. It basically just calls
the function that converts the windows-type "English_New Zealand.1252"
locale name string into, e.g. "en_NZ". Then, since GetNLSVersionEx()
wants yet another variant with a - rather than an _, I've just added a
couple of lines to swap the _ for a -. There's a bit of extra work
there since IsoLocaleName() just did the opposite, so perhaps doing it
that way was lazy of me. I'd have invented some other function if I
could have thought of a meaningful name for it, then just have the ISO
version of it swap - for _.
It would be good if this could also be tested on Visual Studio version
12 as I see IsoLocaleName() does something else for anything before
15. I only have 10 and 17 installed and I see we don't support
anything before 12 on master per:
"Unable to determine Visual Studio version: Visual Studio versions
before 12.0 aren't supported. at
L:/Projects/Postgres/d/src/tools/msvc/Mkvcbuild.pm line 93."
David
Attachments:
fix_locale_versioning_on_windows.patchtext/plain; charset=US-ASCII; name=fix_locale_versioning_on_windows.patchDownload
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 3b0324ce18..4982fc7eb1 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -943,7 +943,7 @@ cache_locale_time(void)
}
-#if defined(WIN32) && defined(LC_MESSAGES)
+#if defined(WIN32)
/*
* Convert a Windows setlocale() argument to a Unix-style one.
*
@@ -1692,6 +1692,16 @@ get_collation_actual_version(char collprovider, const char *collcollate)
*/
NLSVERSIONINFOEX version = {sizeof(NLSVERSIONINFOEX)};
WCHAR wide_collcollate[LOCALE_NAME_MAX_LENGTH];
+ char *shortlocale = IsoLocaleName(collcollate);
+ char *underscore;
+
+ /*
+ * GetNLSVersionEx wants <language>-<REGION> format, whereas the ISO
+ * format is <language>_<REGION>. So replace the _ with -
+ */
+ underscore = strchr(shortlocale, '_');
+ if (underscore)
+ *underscore = '-';
/* These would be invalid arguments, but have no version. */
if (pg_strcasecmp("c", collcollate) == 0 ||
@@ -1699,12 +1709,12 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return NULL;
/* For all other names, ask the OS. */
- MultiByteToWideChar(CP_ACP, 0, collcollate, -1, wide_collcollate,
+ MultiByteToWideChar(CP_ACP, 0, shortlocale, -1, wide_collcollate,
LOCALE_NAME_MAX_LENGTH);
if (!GetNLSVersionEx(COMPARE_STRING, wide_collcollate, &version))
ereport(ERROR,
(errmsg("could not get collation version for locale \"%s\": error code %lu",
- collcollate,
+ shortlocale,
GetLastError())));
collversion = psprintf("%d.%d,%d.%d",
(version.dwNLSVersion >> 8) & 0xFFFF,
Re: Collation versioning
On Tue, Nov 3, 2020 at 1:51 PM David Rowley <dgrowleyml@gmail.com> wrote:
On Tue, 3 Nov 2020 at 12:29, David Rowley <dgrowleyml@gmail.com> wrote:
Running low on ideas for now, so thought I'd post this in case it
someone thinks of something else.FWIW, the attached does fix the issue for me. It basically just calls
the function that converts the windows-type "English_New Zealand.1252"
locale name string into, e.g. "en_NZ". Then, since GetNLSVersionEx()
wants yet another variant with a - rather than an _, I've just added a
couple of lines to swap the _ for a -. There's a bit of extra work
there since IsoLocaleName() just did the opposite, so perhaps doing it
that way was lazy of me. I'd have invented some other function if I
could have thought of a meaningful name for it, then just have the ISO
version of it swap - for _.
Thanks! Hmm, it looks like Windows calls the hyphenated ISO
language-country form a "tag". It makes me slightly nervous to ask
for the version of a transformed name with the encoding stripped, but
it does seem entirely plausible that it gives the answer we seek. I
suppose if we were starting from a clean slate we might want to
perform this transformation up front so that we have it in datcollate
and then not have to think about the older form ever again. If we
decided to do that going forward, the last trace of that problem would
live in pg_upgrade. If we ever extend pg_import_system_collations()
to cover Windows, we should make sure it captures the tag form.
It would be good if this could also be tested on Visual Studio version
12 as I see IsoLocaleName() does something else for anything before
15. I only have 10 and 17 installed and I see we don't support
anything before 12 on master per:
I think others have mentioned that it might be time to drop some older
Windows versions. I don't follow that stuff, so I've quietly added a
name to the CC list and will hope for the best :-)
Re: Collation versioning
On Tue, Nov 3, 2020 at 4:39 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Tue, Nov 3, 2020 at 1:51 PM David Rowley <dgrowleyml@gmail.com> wrote:
It would be good if this could also be tested on Visual Studio version
12 as I see IsoLocaleName() does something else for anything before
15. I only have 10 and 17 installed and I see we don't support
anything before 12 on master per:I think others have mentioned that it might be time to drop some older
Windows versions. I don't follow that stuff, so I've quietly added a
name to the CC list and will hope for the best :-)
There has been some talk about pushing _WIN32_WINNT to newer releases, but
ended without an actual patch for doing so. Maybe we can revisit that in
another thread.
[1]: /messages/by-id/20200218065418.GK4176@paquier.xyz
/messages/by-id/20200218065418.GK4176@paquier.xyz
Regards,
Juan José Santamaría Flecha
Re: Collation versioning
On Tue, Nov 3, 2020 at 4:38 PM Thomas Munro <thomas.munro@gmail.com> wrote:
On Tue, Nov 3, 2020 at 1:51 PM David Rowley <dgrowleyml@gmail.com> wrote:
On Tue, 3 Nov 2020 at 12:29, David Rowley <dgrowleyml@gmail.com> wrote:
Running low on ideas for now, so thought I'd post this in case it
someone thinks of something else.FWIW, the attached does fix the issue for me. It basically just calls
the function that converts the windows-type "English_New Zealand.1252"
locale name string into, e.g. "en_NZ". Then, since GetNLSVersionEx()
wants yet another variant with a - rather than an _, I've just added a
couple of lines to swap the _ for a -. There's a bit of extra work
there since IsoLocaleName() just did the opposite, so perhaps doing it
that way was lazy of me. I'd have invented some other function if I
could have thought of a meaningful name for it, then just have the ISO
version of it swap - for _.Thanks! Hmm, it looks like Windows calls the hyphenated ISO
language-country form a "tag". It makes me slightly nervous to ask
for the version of a transformed name with the encoding stripped, but
it does seem entirely plausible that it gives the answer we seek. I
suppose if we were starting from a clean slate we might want to
perform this transformation up front so that we have it in datcollate
and then not have to think about the older form ever again. If we
decided to do that going forward, the last trace of that problem would
live in pg_upgrade. If we ever extend pg_import_system_collations()
to cover Windows, we should make sure it captures the tag form.
So we have:
1. Windows locale names, like "English_United States.1252". Windows
still returns these from setlocale(), so they finish up in datcollate,
and yet some relevant APIs don't accept them, at least on some
machines.
2. BCP 47/RFC 5646 language tags, like "en-US". Windows uses these
in relevant new APIs, including the case in point.
3. Unix-style (XPG? ISO/IEC 15897?) locale names, like "en_US"
("language[_territory[.codeset]][@modifier]"). These are used for
message catalogues.
We have a VS2015+ way of converting from form 1 to form 2 (and thence
3 by s/-/_/), and an older way. Unfortunately, the new way looks a
little too fuzzy: if i'm reading it right, search_locale_enum() might
stop on either "en" or "en-AU", given "English_Australia", depending
on the search order, no? This may be fine for the purpose of looking
up error messages with gettext() (where there is only one English
language message catalogue, we haven't got around to translating our
errors into 'strayan yet), but it doesn't seem like a good way to look
up the collation version; for all I know, "en" variants might change
independently (I doubt it in practice, but in theory it's wrong). We
want the same algorithm that Windows uses internally to resolve the
old style name to a collation; in other words we probably want
something more like the code path that they took away in VS2015 :-(.
Re: Collation versioning
Thomas Munro <thomas.munro@gmail.com> writes:
On Tue, Nov 3, 2020 at 4:38 PM Thomas Munro <thomas.munro@gmail.com> wrote:
On Tue, Nov 3, 2020 at 1:51 PM David Rowley <dgrowleyml@gmail.com> wrote:
FWIW, the attached does fix the issue for me. It basically just calls
the function that converts the windows-type "English_New Zealand.1252"
locale name string into, e.g. "en_NZ".
We want the same algorithm that Windows uses internally to resolve the
old style name to a collation; in other words we probably want
something more like the code path that they took away in VS2015 :-(.
Yeah. In the short run, though, it'd be nice to un-break the buildfarm.
Maybe we could push David's code or something similar, and then
contemplate better ways at leisure?
regards, tom lane
Re: Collation versioning
On Wed, Nov 4, 2020 at 10:52 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Thomas Munro <thomas.munro@gmail.com> writes:
We want the same algorithm that Windows uses internally to resolve the
old style name to a collation; in other words we probably want
something more like the code path that they took away in VS2015 :-(.Yeah. In the short run, though, it'd be nice to un-break the buildfarm.
Maybe we could push David's code or something similar, and then
contemplate better ways at leisure?
Ok, yeah, I'll do that in the next few hours.
Re: Collation versioning
Re: Thomas Munro
for all I know, "en" variants might change
independently (I doubt it in practice, but in theory it's wrong).
Long before the glibc 2.28 incident, the same collation change
had already happened twice, namely between RedHat 5 -> 6 -> 7, for
de_DE.UTF-8 only. de_AT.UTF-8 and all other locales were unaffected.
At the time I didn't connect the dots to check if Debian was affected
as well, but of course later testing revealed it was since it was a
change in glibc.
(German blogpost: https://www.credativ.de/blog/postgresql/postgresql-und-inkompatible-deutsche-spracheigenschaften-in-centos-rhel/)
Christoph
Re: Collation versioning
On Wed, Nov 4, 2020 at 10:56 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Wed, Nov 4, 2020 at 10:52 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Thomas Munro <thomas.munro@gmail.com> writes:
We want the same algorithm that Windows uses internally to resolve the
old style name to a collation; in other words we probably want
something more like the code path that they took away in VS2015 :-(.Yeah. In the short run, though, it'd be nice to un-break the buildfarm.
Maybe we could push David's code or something similar, and then
contemplate better ways at leisure?Ok, yeah, I'll do that in the next few hours.
I can't bring myself to commit that, it's not really in the spirit of
this data integrity feature, and it's not our business to second guess
the relationship between different locale naming schemes through fuzzy
logic. Instead, I propose to just neuter the feature if Windows
decides it can't understand a locale names that it gave us. It should
still work fine with something like initdb --lc-collate=en-US. Here's
an untested patch. Thoughts?
Attachments:
0001-Tolerate-version-lookup-failure-for-old-style-Window.patchtext/x-patch; charset=US-ASCII; name=0001-Tolerate-version-lookup-failure-for-old-style-Window.patchDownload
From 0e99688adb3c1b18c01fbfd8e2488e5769309fc7 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 4 Nov 2020 13:49:05 +1300
Subject: [PATCH] Tolerate version lookup failure for old-style Windows locale
names.
Accept that we can't get versions for such locale names for now, leaving
it up to the user to provide the modern language tag format to benefit
from the collation versioning feature. It's not clear that we can do
the conversion from the old style to the new style reliably enough for
this purpose.
Unfortunately, this includes the values that are typically installed as
database default by initdb, so it'd be nice to find a better solution
than this.
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/utils/adt/pg_locale.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 3b0324ce18..d5a0169420 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1702,10 +1702,22 @@ get_collation_actual_version(char collprovider, const char *collcollate)
MultiByteToWideChar(CP_ACP, 0, collcollate, -1, wide_collcollate,
LOCALE_NAME_MAX_LENGTH);
if (!GetNLSVersionEx(COMPARE_STRING, wide_collcollate, &version))
+ {
+ /*
+ * GetNLSVersionEx() wants a language tag such as "en-US", not a
+ * locale name like "English_United States.1252". Until those
+ * values can be prevented from entering the system, or 100%
+ * reliably converted to the more useful tag format, tolerate the
+ * resulting error and report that we have no version data.
+ */
+ if (GetLastError() == ERROR_INVALID_PARAMETER)
+ return NULL;
+
ereport(ERROR,
(errmsg("could not get collation version for locale \"%s\": error code %lu",
collcollate,
GetLastError())));
+ }
collversion = psprintf("%d.%d,%d.%d",
(version.dwNLSVersion >> 8) & 0xFFFF,
version.dwNLSVersion & 0xFF,
--
2.20.1
Re: Collation versioning
Thomas Munro <thomas.munro@gmail.com> writes:
I can't bring myself to commit that, it's not really in the spirit of
this data integrity feature, and it's not our business to second guess
the relationship between different locale naming schemes through fuzzy
logic. Instead, I propose to just neuter the feature if Windows
decides it can't understand a locale names that it gave us. It should
still work fine with something like initdb --lc-collate=en-US. Here's
an untested patch. Thoughts?
Works for me, at least as a short-term solution.
regards, tom lane
Re: Collation versioning
On Wed, 4 Nov 2020 at 14:21, Thomas Munro <thomas.munro@gmail.com> wrote:
On Wed, Nov 4, 2020 at 10:56 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Wed, Nov 4, 2020 at 10:52 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Thomas Munro <thomas.munro@gmail.com> writes:
We want the same algorithm that Windows uses internally to resolve the
old style name to a collation; in other words we probably want
something more like the code path that they took away in VS2015 :-(.Yeah. In the short run, though, it'd be nice to un-break the buildfarm.
Maybe we could push David's code or something similar, and then
contemplate better ways at leisure?Ok, yeah, I'll do that in the next few hours.
I can't bring myself to commit that, it's not really in the spirit of
this data integrity feature, and it's not our business to second guess
the relationship between different locale naming schemes through fuzzy
logic. Instead, I propose to just neuter the feature if Windows
decides it can't understand a locale names that it gave us. It should
still work fine with something like initdb --lc-collate=en-US. Here's
an untested patch. Thoughts?
I gave this a quick test.
initdb works fine. I ran vcregress upgradecheck and it passes.
With my default locale of English.New Zealand.1252 I get zero rows from:
select * from pg_depend where coalesce(refobjversion,'') <> '';
if I initdb with --lc-collate=en-NZ, it works and I see:
postgres=# select * from pg_depend where coalesce(refobjversion,'') <> '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
deptype | refobjversion
---------+-------+----------+------------+----------+-------------+---------+-----------------
2606 | 12512 | 0 | 3456 | 100 | 0 | n
| 1538.14,1538.14
(1 row)
David
Re: Collation versioning
On Wed, Nov 4, 2020 at 2:56 PM David Rowley <dgrowleyml@gmail.com> wrote:
initdb works fine. I ran vcregress upgradecheck and it passes.
With my default locale of English.New Zealand.1252 I get zero rows from:
select * from pg_depend where coalesce(refobjversion,'') <> '';
if I initdb with --lc-collate=en-NZ, it works and I see:
postgres=# select * from pg_depend where coalesce(refobjversion,'') <> '';
classid | objid | objsubid | refclassid | refobjid | refobjsubid |
deptype | refobjversion
---------+-------+----------+------------+----------+-------------+---------+-----------------
2606 | 12512 | 0 | 3456 | 100 | 0 | n
| 1538.14,1538.14
(1 row)
Thanks for all the help and testing! Pushed. If we don't come up
with something better I'll need to figure out how to explain this in
the manual. (Will no one rid us of these meddlesome old format locale
names? It seems like pg_locale.c could drop a lot of rather
unpleasant code if initdb, CREATE COLLATION, and CREATE DATABASE
didn't allow them into the catalogue in the first place...)
Re: Collation versioning
On Tue, Nov 3, 2020 at 10:49 PM Thomas Munro <thomas.munro@gmail.com> wrote:
So we have:
1. Windows locale names, like "English_United States.1252". Windows
still returns these from setlocale(), so they finish up in datcollate,
and yet some relevant APIs don't accept them, at least on some
machines.2. BCP 47/RFC 5646 language tags, like "en-US". Windows uses these
in relevant new APIs, including the case in point.3. Unix-style (XPG? ISO/IEC 15897?) locale names, like "en_US"
("language[_territory[.codeset]][@modifier]"). These are used for
message catalogues.We have a VS2015+ way of converting from form 1 to form 2 (and thence
3 by s/-/_/), and an older way. Unfortunately, the new way looks a
little too fuzzy: if i'm reading it right, search_locale_enum() might
stop on either "en" or "en-AU", given "English_Australia", depending
on the search order, no?
No, that is not the case. "English" could match any locale if the
enumeration order was to be changed in the future, right now the order is a
given (Language, Location), but "English_Australia" can only match "en-AU".
This may be fine for the purpose of looking
up error messages with gettext() (where there is only one English
language message catalogue, we haven't got around to translating our
errors into 'strayan yet), but it doesn't seem like a good way to look
up the collation version; for all I know, "en" variants might change
independently (I doubt it in practice, but in theory it's wrong). We
want the same algorithm that Windows uses internally to resolve the
old style name to a collation; in other words we probably want
something more like the code path that they took away in VS2015 :-(.
We could create a static table with the conversion based on what was
discussed for commit a169155, please find attached a spreadsheet with the
comparison. This would require maintenance as new LCIDs are released [1]https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/70feba9f-294e-491e-b6eb-56532684c37f.
[1]: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/70feba9f-294e-491e-b6eb-56532684c37f
https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/70feba9f-294e-491e-b6eb-56532684c37f
Regards,
Juan José Santamaría
Attachments:
WindowsNLSLocales.odsapplication/vnd.oasis.opendocument.spreadsheet; name=WindowsNLSLocales.odsDownload
Re: Collation versioning
On Wed, Nov 04, 2020 at 08:44:15AM +0100, Juan José Santamaría Flecha wrote:
We could create a static table with the conversion based on what was
discussed for commit a169155, please find attached a spreadsheet with the
comparison. This would require maintenance as new LCIDs are released [1].
I am honestly not a fan of something like that as it has good chances
to rot.
--
Michael
Re: Collation versioning
On Wed, Nov 4, 2020 at 4:11 PM Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Nov 04, 2020 at 08:44:15AM +0100, Juan José Santamaría Flecha wrote:
We could create a static table with the conversion based on what was
discussed for commit a169155, please find attached a spreadsheet with the
comparison. This would require maintenance as new LCIDs are released [1].I am honestly not a fan of something like that as it has good chances
to rot.
Same here.
Re: Collation versioning
On Tue, 2020-11-03 at 23:14 +0100, Christoph Berg wrote:
Re: Thomas Munro
for all I know, "en" variants might change
independently (I doubt it in practice, but in theory it's wrong).Long before the glibc 2.28 incident, the same collation change
had already happened twice, namely between RedHat 5 -> 6 -> 7, for
de_DE.UTF-8 only. de_AT.UTF-8 and all other locales were unaffected.At the time I didn't connect the dots to check if Debian was affected
as well, but of course later testing revealed it was since it was a
change in glibc.
Yes, this is a persistent pain; I had several customers suffering from
these issues too.
I wish
/messages/by-id/5e756dd6-0e91-d778-96fd-b1bcb06c161a@2ndquadrant.com
hade made it into core.
Yours,
Laurenz Albe
Re: Collation versioning
On Wed, Nov 4, 2020 at 9:11 PM Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Nov 04, 2020 at 08:44:15AM +0100, Juan José Santamaría Flecha wrote:
We could create a static table with the conversion based on what was
discussed for commit a169155, please find attached a spreadsheet with the
comparison. This would require maintenance as new LCIDs are released [1].I am honestly not a fan of something like that as it has good chances
to rot.
No opinion on that, other than that we'd surely want a machine
readable version. As for *when* we use that information, I'm
wondering if it would make sense to convert datcollate to a language
tag in initdb, and also change pg_upgrade's equivalent_locale()
function to consider "English_United States.*" and "en-US" to be
equivalent when upgrading to 14 (which would then be the only point
you'd ever have to have faith that we can convert the old style names
to the new names correctly). I'm unlikely to work on this myself as I
have other operating systems to fix, but I'll certainly be happy if
somehow we can get versioning for default on Windows in PG14 and not
have to come up with weasel words in the manual.
Just by the way, I think Windows does one thing pretty nicely here: it
has versions with a major and a minor part. If the minor part goes
up, it means that they only added new code points, but didn't change
the ordering of any existing code points, so in some circumstances you
don't have to rebuild (which I think is the case for many Unicode
updates, adding new Chinese characters or emojis or whatever). I
thought about whether we should replace the strcmp() comparison with a
call into provider-specific code, and in the case of Win32 locales it
could maybe understand that. But there are two problems of limited
surmountability: (1) You have an idex built with version 42.1, and now
version 42.3 is present; OK, we can read this index, but if we write
any new data, then a streaming replica that has 42.2 will think it's
OK to read data, but it's not OK; so as soon as you write, you'd need
to update the catalogue, which is quite complicated (cf enum types);
(2) The whole theory only holds together if you didn't actually use
any of the new codepoints introduced by 42.3 while the index said
42.1, yet PostgreSQL isn't validating the codepoints you use against
the collation provider's internal map of valid code points. So I gave
up with that line of thinking for now.
Re: Collation versioning
On Fri, Oct 4, 2019 at 1:25 AM Thomas Munro <thomas.munro@gmail.com> wrote:
Ok. Here's one like that. Also, a WIP patch for FreeBSD.
Here's an updated patch for FreeBSD, which I'll sit on for a bit
longer. It needs bleeding edge 13-CURRENT (due out Q1ish).
Attachments:
0001-Add-collation-versions-for-FreeBSD.patchtext/x-patch; charset=US-ASCII; name=0001-Add-collation-versions-for-FreeBSD.patchDownload
From b9cb5562457c214c48c0a6334b8ed3264f50e3d6 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sun, 8 Nov 2020 21:12:12 +1300
Subject: [PATCH] Add collation versions for FreeBSD.
On FreeBSD 13, use querylocale() to read the current version of libc
collations.
---
doc/src/sgml/charset.sgml | 3 ++-
src/backend/utils/adt/pg_locale.c | 20 ++++++++++++++++++++
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 832a701523..e151987eff 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -973,7 +973,8 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
Version information is available from the
<literal>icu</literal> provider on all operating systems. For the
<literal>libc</literal> provider, versions are currently only available
- on systems using the GNU C library (most Linux systems) and Windows.
+ on systems using the GNU C library (most Linux systems), FreeBSD and
+ Windows.
</para>
<note>
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 1dfe343b79..b1a8eb9a4e 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -1684,6 +1684,26 @@ get_collation_actual_version(char collprovider, const char *collcollate)
/* Use the glibc version because we don't have anything better. */
collversion = pstrdup(gnu_get_libc_version());
+#elif defined(LC_VERSION_MASK)
+ locale_t loc;
+
+ /* C[.encoding] and POSIX never change. */
+ if (strcmp("C", collcollate) == 0 ||
+ strncmp("C.", collcollate, 2) == 0 ||
+ strcmp("POSIX", collcollate) == 0)
+ return NULL;
+
+ /* Look up FreeBSD collation version. */
+ loc = newlocale(LC_COLLATE, collcollate, NULL);
+ if (loc)
+ {
+ collversion =
+ pstrdup(querylocale(LC_COLLATE_MASK | LC_VERSION_MASK, loc));
+ freelocale(loc);
+ }
+ else
+ ereport(ERROR,
+ (errmsg("could not load locale \"%s\"", collcollate)));
#elif defined(WIN32) && _WIN32_WINNT >= 0x0600
/*
* If we are targeting Windows Vista and above, we can ask for a name
--
2.28.0
Re: Collation versioning
On Fri, Nov 6, 2020 at 10:56 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Wed, Nov 4, 2020 at 9:11 PM Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Nov 04, 2020 at 08:44:15AM +0100, Juan José Santamaría Flecha wrote:
We could create a static table with the conversion based on what was
discussed for commit a169155, please find attached a spreadsheet with the
comparison. This would require maintenance as new LCIDs are released [1].
I am honestly not a fan of something like that as it has good chances
to rot.
No opinion on that, other than that we'd surely want a machine
readable version. As for *when* we use that information, I'm
wondering if it would make sense to convert datcollate to a language
tag in initdb, and also change pg_upgrade's equivalent_locale()
function to consider "English_United States.*" and "en-US" to be
equivalent when upgrading to 14 (which would then be the only point
you'd ever have to have faith that we can convert the old style names
to the new names correctly). I'm unlikely to work on this myself as I
have other operating systems to fix, but I'll certainly be happy if
somehow we can get versioning for default on Windows in PG14 and not
have to come up with weasel words in the manual.
FYI I have added this as an open item for PostgreSQL 14. My default
action will be to document this limitation, if we can't come up with
something better in time.
Re: Collation versioning
On Mon, Mar 15, 2021 at 2:25 PM Thomas Munro <thomas.munro@gmail.com> wrote:
FYI I have added this as an open item for PostgreSQL 14. My default
action will be to document this limitation, if we can't come up with
something better in time.
Here is a short doc update to explain the situation on Windows and
close that open item.
PS While trying to find official names to use to refer to the "en-US"
and "English_United States.1252" forms, I came across these sentences
in the Windows documentation[1]https://docs.microsoft.com/en-us/cpp/c-runtime-library/locale-names-languages-and-country-region-strings?view=msvc-160, which support the idea already
discussed of trying to prevent the latter format from ever entering
our catalogs, in some future release:
"The locale-name form is a short, IETF-standardized string; for
example, en-US for English (United States) or bs-Cyrl-BA for Bosnian
(Cyrillic, Bosnia and Herzegovina). These forms are preferred. [...]"
"The language[_country-region[.code-page]] form is stored in the
locale setting for a category when a language string, or language
string and country or region string, is used to create the locale.
[...] We do not recommend this form for locale strings embedded in
code or serialized to storage, because these strings are more likely
to be changed by an operating system update than the locale name
form."
Attachments:
0001-Doc-Document-known-problem-with-Windows-collation-ve.patchtext/x-patch; charset=US-ASCII; name=0001-Doc-Document-known-problem-with-Windows-collation-ve.patchDownload
From fd6c376dba21fdb0020d9b9de08fb878bb66f23d Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 16 Apr 2021 10:21:48 +1200
Subject: [PATCH] Doc: Document known problem with Windows collation versions.
Warn users that locales with traditional Windows NLS names like
"English_United States.1252" won't provide version information, and that
something like initdb --lc-collate=en-US would be needed to fix that
problem for the database default collation.
Discussion: https://postgr.es/m/CA%2BhUKGJ_hk3rU%3D%3Dg2FpAMChb_4i%2BTJacpjjqFsinY-tRM3FBmA%40mail.gmail.com
---
doc/src/sgml/charset.sgml | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 1b00e543a6..9630b18988 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -985,6 +985,15 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
approach is imperfect as maintainers are free to back-port newer
collation definitions to older C library releases.
</para>
+ <para>
+ When using Windows collations, version information is only available for
+ collations defined with IETF BCP47 locale names such as
+ <literal>en-US</literal>. Currently, <command>initdb</command> selects
+ a default locale using a traditional Windows language and country
+ string such as <literal>English_United States.1252</literal>. The
+ <literal>--lc-collate</literal> option can be used to provide an explicit
+ locale name in IETF-standardized form.
+ </para>
</note>
</sect2>
</sect1>
--
2.30.1