locale / encoding / meson cleanup

Hello!

Andrew Dunstan <andrew@dunslane.net> writes:

...
1. test_regex_utf8 depends on the ctype, not just the encoding
...

I never run this test so it was a big surprise to me that after this:

PG_TEST_INITDB_EXTRA_OPTS='--no-locale --encoding=UTF8' meson test -C build/ --suite setup --suite test_regex

The error was there crystal clear! Tested patched 0001 and 0002 and now
the test pass and works perfectly on riscv64 and x86.

2. collate.linux.utf8 has never run on a meson build

While checking whether anything else fails in a C+UTF8 database, I found that
collate.linux.utf8 wasn't running on my build at all. Its guard includes

    version() !~ 'linux-gnu'

and meson builds don't produce that string:

    meson     PostgreSQL 20devel on aarch64-linux, compiled by gcc-13.3.0,
64-bit
    autoconf  PostgreSQL 20devel on aarch64-unknown-linux-gnu, compiled by gcc
...

I was able to reproduce the same issue with a riscv64 and x86, the patch
0003 was enough to fix the issue.

I've run a long test and found some errors, not related to this patch,
but will be triggered with the options:

PG_TEST_INITDB_EXTRA_OPTS='--no-locale --encoding=UTF8'

Don't know if are useful information but here is some information.

Test failed:

222/403 pg_dump - postgresql:pg_dump/010_dump_connstr ERROR 14.13s exit status 4
253/403 scripts - postgresql:scripts/200_connstr ERROR 4.18s exit status 3
294/403 subscription - postgresql:subscription/005_encoding ERROR 8.16s exit status 4

The tests failed because it's passing to initdb the `--encoding`
option twice and `--locale` and then `--no-locale` option, which is
weird, but that's clearly and error because of the extra options, not
related, to the patch, but it could be an issue in the farm since these
test are forcing a `--locale` and `--encoding` option. I think that it's
worth to keep these test in mind when testing

Regards,
--
Jonathan Gonzalez V.
EDB
https://www.enterprisedb.com

Hi,

Thank you for working on this!

I reviewed this patch with GPT's help and confirmed that some findings
are relevant.

On Sun, 23 Aug 2026 at 16:01, Andrew Dunstan <andrew@dunslane.net> wrote:

2. collate.linux.utf8 has never run on a meson build

infinite_recurse is the other test that matches on the platform string,
and it
gets it the other way round: it means to skip itself on ppc64 Linux
because of
a kernel bug, and on a meson build the match will never fire, so it will be
running the case it's meant to stay away from. I don't have a ppc64
machine to
confirm that end of it, so that part is a reading of the code rather than
something I've observed.

It seems meson returns 'ppc64' [1]https://mesonbuild.com/Reference-tables.html#cpu-families.

0004 makes the meson build report the GNU host triplet, by asking the
compiler
for it with -dumpmachine where it supports that and falling back to the
present
behaviour otherwise.

+host_tuple = '@0@-@1@'.format(host_machine.cpu_family(), host_system)
+if cc.get_id() in ['gcc', 'clang']
+  dumpmachine = run_command(cc.cmd_array(), '-dumpmachine', check: false)

I think we need to add c_args here, like:

dumpmachine = run_command(
cc.cmd_array() + get_option('c_args'), '-dumpmachine', check: false)

There could be a '--target' in the c_args which might affect the result [2]https://clang.llvm.org/docs/ClangCommandLineReference.html.

[1]: https://mesonbuild.com/Reference-tables.html#cpu-families
[2]: https://clang.llvm.org/docs/ClangCommandLineReference.html

--
Regards,
Nazir Bilal Yavuz
Microsoft

On 2026-08-23 Su 9:00 AM, Andrew Dunstan wrote:

Hi,

(Astute observers will notice that Claude loves the sound of its own
voice - or my voice which it's trying to emulate - a lot more than I
do. I have trimmed the text quite a bit.)

Tom observed in [1] that nothing in the buildfarm builds a cluster
with locale C and encoding UTF8, that this is where the recent
to_date() crash went undetected, and that the animal configuration had
no way to ask for one. I've taught the buildfarm client to accept an
encoding alongside the locale. However, it's not yet released, because
running an animal that way turned up two things, and the second
explains in part why the first went unnoticed for as long as it did.

1. test_regex_utf8 depends on the ctype, not just the encoding

The file decides whether to run by looking at the encoding alone:

    SELECT getdatabaseencoding() <> 'UTF8' AS skip_test \gset

but two of its cases also depend on the database ctype, so it fails in a
database with encoding UTF8 and locale C:

    @@ -152,7 +152,7 @@
        test_regex
     -----------------
      {0,REG_ULOCALE}
    - {xᔀሷ}
    + {x}
     (2 rows)

     select * from test_regex('[[:lower:]]+',  E'xᔀሷ', 'L');
    @@ -166,7 +166,7 @@
        test_regex
     -----------------
      {0,REG_ULOCALE}
    - {xᔀሷ}
    + {x}
     (2 rows)

The new output is the correct one: under ctype C, isgraph() and
isprint() are
false for anything outside ASCII, so only the x matches. The cases are
[[:graph:]] and [[:print:]] over E'xᔀሷ'. It isn't really about the
regex code;
the same difference shows up in plain SQL in two clusters differing
only in
locale.

Those two are the only ctype-dependent assertions in the file —
everything else
uses explicit code point ranges such as [\u1000-\u2000], or an input
with a
separator (x*, x_*) that ends the match inside ASCII, which is presumably
deliberate.

Patches 0001 (for 15 and 16) and 0002 (for 17+) attached.

0002 gives the two cases an explicit collation:

    select * from test_regex('[[:graph:]]+',  E'xᔀሷ' COLLATE
pg_c_utf8, 'L');
    select * from test_regex('[[:print:]]+',  E'xᔀሷ' COLLATE
pg_c_utf8, 'L');

test_regex.c already threads PG_GET_COLLATION() into the compile, and
pg_c_utf8 exists in every UTF8 database, which is the only place this
file
runs. That returns {xᔀሷ} in a C+UTF8 database and in en_US.utf8, so
the result
lines in the expected file don't change at all — only the echoed query
text
does. It seems to me strictly better than what's there now, since the
cases
stop depending on how the animal happened to be initdb'd.

On 15 and 16 there's no collation to point at — ucs_basic has
collctype C, the builtin provider is 17+, and ICU depends on the build
— so 0001 just adds a second expected file with the C-ctype answers,
the way json_encoding.sql does for its two encodings.

I did consider just neutralising the two inputs, by putting a space or
a tab in
front of the non-ASCII characters the way x* and x_* already do
elsewhere in
that block. It works and it backpatches everywhere. I'd be sorry to do
it,
though: those two cases are the only ones in the file that exercise
Unicode
ctype at all, and after such a change they would pass even if the
ctype lookup
for non-ASCII were completely broken.

I have pushed these two. That's enough to unblock the buildfarm work.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On 2026-08-26 We 10:08 AM, Nazir Bilal Yavuz wrote:

Hi,

Thank you for working on this!

I reviewed this patch with GPT's help and confirmed that some findings
are relevant.

On Sun, 23 Aug 2026 at 16:01, Andrew Dunstan <andrew@dunslane.net> wrote:

2. collate.linux.utf8 has never run on a meson build

infinite_recurse is the other test that matches on the platform string,
and it
gets it the other way round: it means to skip itself on ppc64 Linux
because of
a kernel bug, and on a meson build the match will never fire, so it will be
running the case it's meant to stay away from. I don't have a ppc64
machine to
confirm that end of it, so that part is a reading of the code rather than
something I've observed.

It seems meson returns 'ppc64' [1].

0004 makes the meson build report the GNU host triplet, by asking the
compiler
for it with -dumpmachine where it supports that and falling back to the
present
behaviour otherwise.

+host_tuple = '@0@-@1@'.format(host_machine.cpu_family(), host_system)
+if cc.get_id() in ['gcc', 'clang']
+  dumpmachine = run_command(cc.cmd_array(), '-dumpmachine', check: false)

I think we need to add c_args here, like:

dumpmachine = run_command(
cc.cmd_array() + get_option('c_args'), '-dumpmachine', check: false)

There could be a '--target' in the c_args which might affect the result [2].

[1] https://mesonbuild.com/Reference-tables.html#cpu-families
[2] https://clang.llvm.org/docs/ClangCommandLineReference.html

Thanks for checking. I adjusted patch 3 accordingly and have pushed it.

I'll rework patch 4 according to your suggestions and add a CF entry for it.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On 2026-08-26 We 10:08 AM, Nazir Bilal Yavuz wrote:

0004 makes the meson build report the GNU host triplet, by asking the
compiler
for it with -dumpmachine where it supports that and falling back to the
present
behaviour otherwise.

+host_tuple = '@0@-@1@'.format(host_machine.cpu_family(), host_system)
+if cc.get_id() in ['gcc', 'clang']
+  dumpmachine = run_command(cc.cmd_array(), '-dumpmachine', check: false)

I think we need to add c_args here, like:

dumpmachine = run_command(
cc.cmd_array() + get_option('c_args'), '-dumpmachine', check: false)

There could be a '--target' in the c_args which might affect the result [2].

[2] https://clang.llvm.org/docs/ClangCommandLineReference.html

Here's a reworked patch 4. I asked Claude for a review of the changes
and inconsistencies, and got this:

 What it fixes:
  - The original bug (meson's cpu_family-system never carrying the ABI
suffix) — fixed for gcc/clang builds.
  - Bilal's c_args/--target gap — if you pass -Dc_args=--target=...,
the reported triplet now reflects the actual compile target instead of
the compiler's untargeted default.

  What it doesn't fully close:

  1. gcc vs. clang now disagree with each other on the same host. I
verified this directly during review: gcc -dumpmachine →
aarch64-linux-gnu, clang -dumpmachine → aarch64-unknown-linux-gnu. Both
satisfy the
  two tests' regexes, so nothing breaks functionally, but version()'s
platform substring is no longer mechanically identical across compilers
on identical hardware — before this patch it was (always
  cpu_family-system, compiler-independent, just wrong). That's a new,
narrower form of inconsistency this patch introduces as a side effect of
the fix.
  2. Meson vs. autoconf agreement isn't structurally guaranteed, only
empirically true for ordinary distro toolchains. Autoconf's string comes
from config.guess/config.sub's own normalization; -dumpmachine
  reports whatever triple the specific compiler was built to report.
They usually coincide because distro gcc/clang packages are typically
built with --target equal to their config.guess triple — but a
  custom-built compiler with an unusual target string would make meson
and autoconf diverge on the identical host, since nothing here
normalizes -dumpmachine's output against GNU's canonical form.
  3. Non-gcc/clang meson builds (MSVC, etc.) still fall back to the old
cpu_family-system format — unchanged, no regression, but also no
consistency gain there. There's no autoconf reference point for MSVC
  anyway, so this doesn't matter in practice.

  So: real inconsistencies removed (the two test guards, the --target
blind spot), one new narrow one accepted as a trade-off
(compiler-dependent vendor field), and one that was never a hard
guarantee to begin
  with (meson vs. autoconf on unusual toolchains).

So, two questions: do we want this at all? After all, we have lived with
the inconsistencies for a while with few ill effects. And if we do, do
we want it for release 19?

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On 29.08.26 00:07, Andrew Dunstan wrote:

So, two questions: do we want this at all? After all, we have lived with
the inconsistencies for a while with few ill effects. And if we do, do
we want it for release 19?

I don't see the point of this. There is nothing that says that
PG_VERSION_STR needs to follow a specific format. It's just an
information string.

On 2026-08-28 Fr 6:18 PM, Peter Eisentraut wrote:

On 29.08.26 00:07, Andrew Dunstan wrote:

So, two questions: do we want this at all? After all, we have lived
with the inconsistencies for a while with few ill effects. And if we
do, do we want it for release 19?

I don't see the point of this.  There is nothing that says that
PG_VERSION_STR needs to follow a specific format.  It's just an
information string.

Well, we had tests relying incorrectly on it, so it's not just an opaque
piece of text. I've fixed those to take account of the variance.

But I'm ok with leaving things as they are.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com