[PROPOSAL] Skip test citext_utf8 on Windows
Greetings, everyone!
While running "installchecks" on databases with UTF-8 encoding the test
citext_utf8 fails because of Turkish dotted I like this:
SELECT 'i'::citext = 'İ'::citext AS t;
t
---
- t
+ f
(1 row)I tried to replicate the test's results by hand and with any collation
that I tried (including --locale="Turkish") this test failed
Also an interesing result of my tesing. If you initialize you DB
with -E utf-8 --locale="Turkish" and then run select LOWER('İ');
the output will be this:
lower
-------
İ
(1 row)
Which I find strange since lower() uses collation that was passed
(default in this case but still)
My PostgreSQL version is this:
postgres=# select version();
version
----------------------------------------------------------------------
PostgreSQL 17devel on x86_64-windows, compiled by gcc-13.1.0, 64-bit
The proposed patch for skipping test is attached
Oleg Tselebrovskiy, Postgres Pro
Attachments:
skip_citext_utf8.patchtext/x-diff; name=skip_citext_utf8.patchDownload
��d i f f - - g i t a / c o n t r i b / c i t e x t / e x p e c t e d / c i t e x t _ u t f 8 . o u t b / c o n t r i b / c i t e x t / e x p e c t e d / c i t e x t _ u t f 8 . o u t
i n d e x 5 d 9 8 8 d c d 4 8 5 . . 6 c 4 0 6 9 f 9 4 6 9 1 0 0 6 4 4
- - - a / c o n t r i b / c i t e x t / e x p e c t e d / c i t e x t _ u t f 8 . o u t
+ + + b / c o n t r i b / c i t e x t / e x p e c t e d / c i t e x t _ u t f 8 . o u t
@ @ - 1 0 , 7 + 1 0 , 8 @ @
S E L E C T g e t d a t a b a s e e n c o d i n g ( ) < > ' U T F 8 ' O R
( S E L E C T ( d a t l o c p r o v i d e r = ' c ' A N D d a t c t y p e = ' C ' ) O R d a t l o c p r o v i d e r = ' i '
F R O M p g _ d a t a b a s e
- W H E R E d a t n a m e = c u r r e n t _ d a t a b a s e ( ) )
+ W H E R E d a t n a m e = c u r r e n t _ d a t a b a s e ( ) ) O R
+ ( v e r s i o n ( ) ~ ' w i n d o w s ' O R v e r s i o n ( ) ~ ' V i s u a l C \ + \ + ' O R v e r s i o n ( ) ~ ' m i n g w 3 2 ' )
A S s k i p _ t e s t \ g s e t
\ i f : s k i p _ t e s t
\ q u i t
d i f f - - g i t a / c o n t r i b / c i t e x t / e x p e c t e d / c i t e x t _ u t f 8 _ 1 . o u t b / c o n t r i b / c i t e x t / e x p e c t e d / c i t e x t _ u t f 8 _ 1 . o u t
i n d e x 7 0 6 5 a 5 d a 1 9 0 . . d 4 4 7 2 b 1 c 3 6 a 1 0 0 6 4 4
- - - a / c o n t r i b / c i t e x t / e x p e c t e d / c i t e x t _ u t f 8 _ 1 . o u t
+ + + b / c o n t r i b / c i t e x t / e x p e c t e d / c i t e x t _ u t f 8 _ 1 . o u t
@ @ - 1 0 , 7 + 1 0 , 8 @ @
S E L E C T g e t d a t a b a s e e n c o d i n g ( ) < > ' U T F 8 ' O R
( S E L E C T ( d a t l o c p r o v i d e r = ' c ' A N D d a t c t y p e = ' C ' ) O R d a t l o c p r o v i d e r = ' i '
F R O M p g _ d a t a b a s e
- W H E R E d a t n a m e = c u r r e n t _ d a t a b a s e ( ) )
+ W H E R E d a t n a m e = c u r r e n t _ d a t a b a s e ( ) ) O R
+ ( v e r s i o n ( ) ~ ' w i n d o w s ' O R v e r s i o n ( ) ~ ' V i s u a l C \ + \ + ' O R v e r s i o n ( ) ~ ' m i n g w 3 2 ' )
A S s k i p _ t e s t \ g s e t
\ i f : s k i p _ t e s t
\ q u i t
d i f f - - g i t a / c o n t r i b / c i t e x t / s q l / c i t e x t _ u t f 8 . s q l b / c o n t r i b / c i t e x t / s q l / c i t e x t _ u t f 8 . s q l
i n d e x 3 4 b 2 3 2 d 6 4 e 2 . . 5 3 7 7 5 c d c d 3 5 1 0 0 6 4 4
- - - a / c o n t r i b / c i t e x t / s q l / c i t e x t _ u t f 8 . s q l
+ + + b / c o n t r i b / c i t e x t / s q l / c i t e x t _ u t f 8 . s q l
@ @ - 1 1 , 7 + 1 1 , 8 @ @
S E L E C T g e t d a t a b a s e e n c o d i n g ( ) < > ' U T F 8 ' O R
( S E L E C T ( d a t l o c p r o v i d e r = ' c ' A N D d a t c t y p e = ' C ' ) O R d a t l o c p r o v i d e r = ' i '
F R O M p g _ d a t a b a s e
- W H E R E d a t n a m e = c u r r e n t _ d a t a b a s e ( ) )
+ W H E R E d a t n a m e = c u r r e n t _ d a t a b a s e ( ) ) O R
+ ( v e r s i o n ( ) ~ ' w i n d o w s ' O R v e r s i o n ( ) ~ ' V i s u a l C \ + \ + ' O R v e r s i o n ( ) ~ ' m i n g w 3 2 ' )
A S s k i p _ t e s t \ g s e t
\ i f : s k i p _ t e s t
\ q u i t
On Mon, Mar 11, 2024 at 03:21:11PM +0700, Oleg Tselebrovskiy wrote:
The proposed patch for skipping test is attached
Your attached patch seems to be in binary format.
--
Michael
On 2024-03-11 Mo 04:21, Oleg Tselebrovskiy wrote:
Greetings, everyone!
While running "installchecks" on databases with UTF-8 encoding the test
citext_utf8 fails because of Turkish dotted I like this:SELECT 'i'::citext = 'İ'::citext AS t;
t
---
- t
+ f
(1 row)I tried to replicate the test's results by hand and with any collation
that I tried (including --locale="Turkish") this test failedAlso an interesing result of my tesing. If you initialize you DB
with -E utf-8 --locale="Turkish" and then run select LOWER('İ');
the output will be this:
lower
-------
İ
(1 row)Which I find strange since lower() uses collation that was passed
(default in this case but still)
Wouldn't we be better off finding a Windows fix for this, instead of
sweeping it under the rug?
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On Tue, Mar 12, 2024 at 2:56 PM Andrew Dunstan <andrew@dunslane.net> wrote:
On 2024-03-11 Mo 04:21, Oleg Tselebrovskiy wrote:
Greetings, everyone!
While running "installchecks" on databases with UTF-8 encoding the test
citext_utf8 fails because of Turkish dotted I like this:SELECT 'i'::citext = 'İ'::citext AS t; t --- - t + f (1 row)I tried to replicate the test's results by hand and with any collation
that I tried (including --locale="Turkish") this test failedAlso an interesing result of my tesing. If you initialize you DB
with -E utf-8 --locale="Turkish" and then run select LOWER('İ');
the output will be this:
lower
-------
İ
(1 row)Which I find strange since lower() uses collation that was passed
(default in this case but still)Wouldn't we be better off finding a Windows fix for this, instead of
sweeping it under the rug?
Given the sorry state of our Windows locale support, I've started
wondering about deleting it and telling users to adopt our nascent
built-in support or ICU[1]/messages/by-id/CA+hUKGJhV__g_TJ0jVqPbnTuqT++M6KFv2wj+9AV-cABNCXN6Q@mail.gmail.com.
This other thread [2]/messages/by-id/1407a2c0-062b-4e4c-b728-438fdff5cb07@manitou-mail.org says the sorting is intransitive so I don't
think it really meets our needs anyway.
[1]: /messages/by-id/CA+hUKGJhV__g_TJ0jVqPbnTuqT++M6KFv2wj+9AV-cABNCXN6Q@mail.gmail.com
[2]: /messages/by-id/1407a2c0-062b-4e4c-b728-438fdff5cb07@manitou-mail.org
Michael Paquier писал(а) 2024-03-12 06:24:
On Mon, Mar 11, 2024 at 03:21:11PM +0700, Oleg Tselebrovskiy wrote:
The proposed patch for skipping test is attached
Your attached patch seems to be in binary format.
--
Michael
Right, I had it saved in not-UTF-8 encoding. Kind of ironic
Here's a fixed version
Attachments:
v2_skip_citext_utf8.patchtext/x-diff; name=v2_skip_citext_utf8.patchDownload
diff --git a/contrib/citext/expected/citext_utf8.out b/contrib/citext/expected/citext_utf8.out
index 5d988dcd485..6c4069f9469 100644
--- a/contrib/citext/expected/citext_utf8.out
+++ b/contrib/citext/expected/citext_utf8.out
@@ -10,7 +10,8 @@
SELECT getdatabaseencoding() <> 'UTF8' OR
(SELECT (datlocprovider = 'c' AND datctype = 'C') OR datlocprovider = 'i'
FROM pg_database
- WHERE datname=current_database())
+ WHERE datname=current_database()) OR
+ (version() ~ 'windows' OR version() ~ 'Visual C\+\+' OR version() ~ 'mingw32')
AS skip_test \gset
\if :skip_test
\quit
diff --git a/contrib/citext/expected/citext_utf8_1.out b/contrib/citext/expected/citext_utf8_1.out
index 7065a5da190..d4472b1c36a 100644
--- a/contrib/citext/expected/citext_utf8_1.out
+++ b/contrib/citext/expected/citext_utf8_1.out
@@ -10,7 +10,8 @@
SELECT getdatabaseencoding() <> 'UTF8' OR
(SELECT (datlocprovider = 'c' AND datctype = 'C') OR datlocprovider = 'i'
FROM pg_database
- WHERE datname=current_database())
+ WHERE datname=current_database()) OR
+ (version() ~ 'windows' OR version() ~ 'Visual C\+\+' OR version() ~ 'mingw32')
AS skip_test \gset
\if :skip_test
\quit
diff --git a/contrib/citext/sql/citext_utf8.sql b/contrib/citext/sql/citext_utf8.sql
index 34b232d64e2..53775cdcd35 100644
--- a/contrib/citext/sql/citext_utf8.sql
+++ b/contrib/citext/sql/citext_utf8.sql
@@ -11,7 +11,8 @@
SELECT getdatabaseencoding() <> 'UTF8' OR
(SELECT (datlocprovider = 'c' AND datctype = 'C') OR datlocprovider = 'i'
FROM pg_database
- WHERE datname=current_database())
+ WHERE datname=current_database()) OR
+ (version() ~ 'windows' OR version() ~ 'Visual C\+\+' OR version() ~ 'mingw32')
AS skip_test \gset
\if :skip_test
\quit
On 2024-03-11 Mo 22:50, Thomas Munro wrote:
On Tue, Mar 12, 2024 at 2:56 PM Andrew Dunstan <andrew@dunslane.net> wrote:
On 2024-03-11 Mo 04:21, Oleg Tselebrovskiy wrote:
Greetings, everyone!
While running "installchecks" on databases with UTF-8 encoding the test
citext_utf8 fails because of Turkish dotted I like this:SELECT 'i'::citext = 'İ'::citext AS t; t --- - t + f (1 row)I tried to replicate the test's results by hand and with any collation
that I tried (including --locale="Turkish") this test failedAlso an interesing result of my tesing. If you initialize you DB
with -E utf-8 --locale="Turkish" and then run select LOWER('İ');
the output will be this:
lower
-------
İ
(1 row)Which I find strange since lower() uses collation that was passed
(default in this case but still)Wouldn't we be better off finding a Windows fix for this, instead of
sweeping it under the rug?Given the sorry state of our Windows locale support, I've started
wondering about deleting it and telling users to adopt our nascent
built-in support or ICU[1].This other thread [2] says the sorting is intransitive so I don't
think it really meets our needs anyway.[1] /messages/by-id/CA+hUKGJhV__g_TJ0jVqPbnTuqT++M6KFv2wj+9AV-cABNCXN6Q@mail.gmail.com
[2] /messages/by-id/1407a2c0-062b-4e4c-b728-438fdff5cb07@manitou-mail.org
Makes more sense than just hacking the tests to avoid running them on
Windows. (I also didn't much like doing it by parsing the version
string, although I know there's at least one precedent for doing that.)
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com