buildfarm + meson

On Wed, Feb 22, 2023 at 06:23:44PM -0500, Andrew Dunstan wrote:

On my Linux box (Fedora 37, where crake runs) I can get a complete run.
There is work to do to make sure we pick up the right log files, and maybe
adjust a module or two. I have adopted a design where instead of trying to
know a lot about the testing regime the client needs to know a lot less.
Instead, it gets meson to tell it the set of tests. I will probably work on
enabling some sort of filter, but I think this makes things more
future-proof. I have stuck with the design of making testing fairly
fine-grained, so each suite runs separately.

Nice!

On a Windows instance, fairly similar to what's running drongo, I can get a
successful build with meson+VS2019, but I'm getting an error in the
regression tests, which don't like setting lc_time to 'de_DE'. Not sure
what's going on there.

What's the regression issue? Some text-field ordering that ought to
be enforced with a C collation?
--
Michael

Hi,

On 2023-02-22 18:23:44 -0500, Andrew Dunstan wrote:

Here's a progress report on adapting the buildfarm client to meson

There is a development branch where I'm working on the changes. They can be
seen here:

<https://github.com/PGBuildFarm/client-code/compare/main...dev/meson&gt;

On my Linux box (Fedora 37, where crake runs) I can get a complete run.

Nice!

There is work to do to make sure we pick up the right log files, and maybe
adjust a module or two. I have adopted a design where instead of trying to
know a lot about the testing regime the client needs to know a lot less.
Instead, it gets meson to tell it the set of tests. I will probably work on
enabling some sort of filter, but I think this makes things more
future-proof. I have stuck with the design of making testing fairly
fine-grained, so each suite runs separately.

I don't understand why you'd want to run each suite separately. Serially
executing the test takes way longer than doing so in parallel. Why would we
want to enforce that?

Particularly because with meson the tests log files and the failed tests can
directly be correlated? And it should be easy to figure out which log files
need to be kept, you can just skip the directories in testrun/ that contain
test.success.

On a Windows instance, fairly similar to what's running drongo, I can get a
successful build with meson+VS2019, but I'm getting an error in the
regression tests, which don't like setting lc_time to 'de_DE'. Not sure
what's going on there.

Huh, that's odd.

meson apparently wants touch and cp installed, although I can't see why at
first glance. For Windows I just copied them into the path from an msys2
installation.

Those should probably be fixed.

Greetings,

Andres Freund

On 2023-02-22 We 19:23, Michael Paquier wrote:

On a Windows instance, fairly similar to what's running drongo, I can get a
successful build with meson+VS2019, but I'm getting an error in the
regression tests, which don't like setting lc_time to 'de_DE'. Not sure
what's going on there.

What's the regression issue? Some text-field ordering that ought to
be enforced with a C collation?

Here's the diff

diff -w -U3 C:/prog/bf/buildroot/HEAD/pgsql/src/test/regress/expected/collate.windows.win1252.out C:/prog/bf/buildroot/HEAD/pgsql.build/testrun/regress/regress/results/collate.windows.win1252.out
--- C:/prog/bf/buildroot/HEAD/pgsql/src/test/regress/expected/collate.windows.win1252.out    2023-02-22 16:32:03.762370300 +0000
+++ C:/prog/bf/buildroot/HEAD/pgsql.build/testrun/regress/regress/results/collate.windows.win1252.out    2023-02-22 22:54:59.281395200 +0000
@@ -363,16 +363,17 @@

  -- to_char
  SET lc_time TO 'de_DE';
+ERROR:  invalid value for parameter "lc_time": "de_DE"
  SELECT to_char(date '2010-03-01', 'DD TMMON YYYY');
     to_char
  -------------
- 01 MRZ 2010
+ 01 MAR 2010
  (1 row)

 SELECT to_char(date '2010-03-01', 'DD TMMON YYYY' COLLATE "de_DE");
    to_char
 -------------
- 01 MRZ 2010
+ 01 MAR 2010
 (1 row)

 -- to_date

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

On 2023-02-22 We 20:20, Andres Freund wrote:

There is work to do to make sure we pick up the right log files, and maybe
adjust a module or two. I have adopted a design where instead of trying to
know a lot about the testing regime the client needs to know a lot less.
Instead, it gets meson to tell it the set of tests. I will probably work on
enabling some sort of filter, but I think this makes things more
future-proof. I have stuck with the design of making testing fairly
fine-grained, so each suite runs separately.

I don't understand why you'd want to run each suite separately. Serially
executing the test takes way longer than doing so in parallel. Why would we
want to enforce that?

Particularly because with meson the tests log files and the failed tests can
directly be correlated? And it should be easy to figure out which log files
need to be kept, you can just skip the directories in testrun/ that contain
test.success.

We can revisit that later. For now I'm more concerned with getting a
working setup. The requirements of the buildfarm are a bit different
from those of a developer, though. Running things in parallel can make
things faster, but that can also increase the compute load. Also,
running things serially makes it easier to report a failure stage that
pinpoints the test that encountered an issue. But like I say we can come
back to this.

On a Windows instance, fairly similar to what's running drongo, I can get a
successful build with meson+VS2019, but I'm getting an error in the
regression tests, which don't like setting lc_time to 'de_DE'. Not sure
what's going on there.

Huh, that's odd.

See my reply to Michael for details

meson apparently wants touch and cp installed, although I can't see why at
first glance. For Windows I just copied them into the path from an msys2
installation.

Those should probably be fixed.

Yeah. For touch I think we can probably just get rid of this line in the
root meson.build:

touch = find_program('touch', native: true)

For cp there doesn't seem to be a formal requirement, but there is a
recipe in src/common/unicode/meson.build that uses it, maybe that's what
caused the failure. On Windows/msvc we could just use copy instead, I think.

I haven't experimented with any of this.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

On 2023-02-23 06:27:23 -0500, Andrew Dunstan wrote:

On 2023-02-22 We 20:20, Andres Freund wrote:

There is work to do to make sure we pick up the right log files, and maybe
adjust a module or two. I have adopted a design where instead of trying to
know a lot about the testing regime the client needs to know a lot less.
Instead, it gets meson to tell it the set of tests. I will probably work on
enabling some sort of filter, but I think this makes things more
future-proof. I have stuck with the design of making testing fairly
fine-grained, so each suite runs separately.

I don't understand why you'd want to run each suite separately. Serially
executing the test takes way longer than doing so in parallel. Why would we
want to enforce that?

Particularly because with meson the tests log files and the failed tests can
directly be correlated? And it should be easy to figure out which log files
need to be kept, you can just skip the directories in testrun/ that contain
test.success.

We can revisit that later. For now I'm more concerned with getting a working
setup.

My fear is that this ends up being entrenched in the design and hard to change
later.

The requirements of the buildfarm are a bit different from those of a
developer, though. Running things in parallel can make things faster, but
that can also increase the compute load.

Sure, I'm not advocating to using a [high] concurrency by default.

Also, running things serially makes it easier to report a failure stage that
pinpoints the test that encountered an issue.

You're relying on running tests in a specific order. Instead you can also just
run tests in parallel and check test status in order and report the first
failed test in that order.

But like I say we can come
back to this.

On a Windows instance, fairly similar to what's running drongo, I can get a
successful build with meson+VS2019, but I'm getting an error in the
regression tests, which don't like setting lc_time to 'de_DE'. Not sure
what's going on there.

Huh, that's odd.

See my reply to Michael for details

I suspect the issue might be related to this:

+               local %ENV = (PATH => $ENV{PATH}, PGUSER => $ENV{PGUSER});
+               @makeout=run_log("meson test --logbase checklog --print-errorlogs --no-rebuild -C $pgsql --suite setup --suite regress");

Greetings,

Andres Freund

On 2023-02-23 Th 10:58, Andres Freund wrote:

On a Windows instance, fairly similar to what's running drongo, I can get a
successful build with meson+VS2019, but I'm getting an error in the
regression tests, which don't like setting lc_time to 'de_DE'. Not sure
what's going on there.

Huh, that's odd.

See my reply to Michael for details

I suspect the issue might be related to this:

+               local %ENV = (PATH => $ENV{PATH}, PGUSER => $ENV{PGUSER});
+               @makeout=run_log("meson test --logbase checklog --print-errorlogs --no-rebuild -C $pgsql --suite setup --suite regress");

I commented out the 'local %ENV' line and still got the error. I also
got the same error running by hand.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

On 2023-02-23 Th 16:12, Andrew Dunstan wrote:

On 2023-02-23 Th 10:58, Andres Freund wrote:

On a Windows instance, fairly similar to what's running drongo, I can get a
successful build with meson+VS2019, but I'm getting an error in the
regression tests, which don't like setting lc_time to 'de_DE'. Not sure
what's going on there.

Huh, that's odd.

See my reply to Michael for details

I suspect the issue might be related to this:

+               local %ENV = (PATH => $ENV{PATH}, PGUSER => $ENV{PGUSER});
+               @makeout=run_log("meson test --logbase checklog --print-errorlogs --no-rebuild -C $pgsql --suite setup --suite regress");

I commented out the 'local %ENV' line and still got the error. I also
got the same error running by hand.

On drongo, this test isn't failing, and I think the reason is that it
runs "make NO_LOCALE=1 check" so it never gets a database with win1252
encoding.

I'm going to try adding a win1252 test to drongo's locales.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

On Fri, Feb 24, 2023 at 2:22 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On drongo, this test isn't failing, and I think the reason is that it runs
"make NO_LOCALE=1 check" so it never gets a database with win1252 encoding.

I'm going to try adding a win1252 test to drongo's locales.

What seems to be failing is the setlocale() for 'de_DE'. I haven't been
able to reproduce it locally, but I've seen something similar reported for
python [1]https://bugs.python.org/issue36792.

As a workaround, can you please test "SET lc_time TO 'de-DE';"?

[1]: https://bugs.python.org/issue36792

Regards,

Juan José Santamaría Flecha

On 2023-02-23 Th 10:58, Andres Freund wrote:

On 2023-02-23 06:27:23 -0500, Andrew Dunstan wrote:

On 2023-02-22 We 20:20, Andres Freund wrote:

There is work to do to make sure we pick up the right log files, and maybe
adjust a module or two. I have adopted a design where instead of trying to
know a lot about the testing regime the client needs to know a lot less.
Instead, it gets meson to tell it the set of tests. I will probably work on
enabling some sort of filter, but I think this makes things more
future-proof. I have stuck with the design of making testing fairly
fine-grained, so each suite runs separately.

I don't understand why you'd want to run each suite separately. Serially
executing the test takes way longer than doing so in parallel. Why would we
want to enforce that?

Particularly because with meson the tests log files and the failed tests can
directly be correlated? And it should be easy to figure out which log files
need to be kept, you can just skip the directories in testrun/ that contain
test.success.

We can revisit that later. For now I'm more concerned with getting a working
setup.

My fear is that this ends up being entrenched in the design and hard to change
later.

The requirements of the buildfarm are a bit different from those of a
developer, though. Running things in parallel can make things faster, but
that can also increase the compute load.

Sure, I'm not advocating to using a [high] concurrency by default.

Perhaps the latest version will be more to your taste. This is now
working on my MSVC test rig (WS2019, VS2019, Strawberry Perl), including
TAP tests. I do get a whole lot of annoying messages like this:

Unknown TAP version. The first line MUST be `TAP version <int>`.
Assuming version 12.

Anyway, I think this is ready for any brave soul who wants to take it
for a test run, not on a reporting animal just yet, though. To activate
it you need the config to have 'using_meson => 1' and a meson_opts
section - see the sample file. You can get the dev/meson version at
<https://github.com/PGBuildFarm/client-code/archive/refs/heads/dev/meson.zip&gt;

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

Hi,

On 2023-03-01 16:21:32 -0500, Andrew Dunstan wrote:

Perhaps the latest version will be more to your taste.

I'll check it out.

This is now working
on my MSVC test rig (WS2019, VS2019, Strawberry Perl), including TAP tests.
I do get a whole lot of annoying messages like this:

Unknown TAP version. The first line MUST be `TAP version <int>`. Assuming
version 12.

The newest minor version has fixed that, it was a misunderstanding about /
imprecision in the tap 14 specification.

Greetings,

Andres Freund

On 2023-03-01 We 16:32, Andres Freund wrote:

This is now working
on my MSVC test rig (WS2019, VS2019, Strawberry Perl), including TAP tests.
I do get a whole lot of annoying messages like this:

Unknown TAP version. The first line MUST be `TAP version <int>`. Assuming
version 12.

The newest minor version has fixed that, it was a misunderstanding about /
imprecision in the tap 14 specification.

Unfortunately, meson v 1.0.1 appears to be broken on Windows, I had to
downgrade back to 1.0.0.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

Hi

On 2023-03-02 17:00:47 -0500, Andrew Dunstan wrote:

On 2023-03-01 We 16:32, Andres Freund wrote:

This is now working
on my MSVC test rig (WS2019, VS2019, Strawberry Perl), including TAP tests.
I do get a whole lot of annoying messages like this:

Unknown TAP version. The first line MUST be `TAP version <int>`. Assuming
version 12.

The newest minor version has fixed that, it was a misunderstanding about /
imprecision in the tap 14 specification.

Unfortunately, meson v 1.0.1 appears to be broken on Windows, I had to
downgrade back to 1.0.0.

Is it possible that you're using a PG checkout from a few days ago? A
hack I used was invalidated by 1.0.1, but I fixed that already.

CI is running with 1.0.1:
https://cirrus-ci.com/task/5806561726038016?logs=configure#L8

Greetings,

Andres Freund

On 2023-03-02 Th 17:06, Andres Freund wrote:

Hi

On 2023-03-02 17:00:47 -0500, Andrew Dunstan wrote:

On 2023-03-01 We 16:32, Andres Freund wrote:

This is now working
on my MSVC test rig (WS2019, VS2019, Strawberry Perl), including TAP tests.
I do get a whole lot of annoying messages like this:

Unknown TAP version. The first line MUST be `TAP version <int>`. Assuming
version 12.

The newest minor version has fixed that, it was a misunderstanding about /
imprecision in the tap 14 specification.

Unfortunately, meson v 1.0.1 appears to be broken on Windows, I had to
downgrade back to 1.0.0.

Is it possible that you're using a PG checkout from a few days ago? A
hack I used was invalidated by 1.0.1, but I fixed that already.

CI is running with 1.0.1:
https://cirrus-ci.com/task/5806561726038016?logs=configure#L8

No, running against PG master tip. I'll get some details - it's not too
hard to switch back and forth.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

Hi,

On 2023-03-01 13:32:58 -0800, Andres Freund wrote:

On 2023-03-01 16:21:32 -0500, Andrew Dunstan wrote:

Perhaps the latest version will be more to your taste.

I'll check it out.

A simple conversion from an existing config failed with:
Can't use an undefined value as an ARRAY reference at /home/bf/src/pgbuildfarm-client-meson/PGBuild/Modules/TestICU.pm line 37.

I disabled TestICU and was able to progress past that.

...
piculet-meson:HEAD [19:12:48] setting up db cluster (C)...
piculet-meson:HEAD [19:12:48] starting db (C)...
piculet-meson:HEAD [19:12:48] running installcheck (C)...
piculet-meson:HEAD [19:12:57] restarting db (C)...
piculet-meson:HEAD [19:12:59] running meson misc installchecks (C) ...
Branch: HEAD
Stage delay_executionInstallCheck-C failed with status 1

The failures are like this:

+ERROR:  extension "dummy_index_am" is not available
+DETAIL:  Could not open extension control file "/home/bf/bf-build/piculet-meson/HEAD/inst/share/postgresql/extension/dummy_index_am.control": No such file or directory.
+HINT:  The extension must first be installed on the system where PostgreSQL is running.

I assume this is in an interaction with b6a0d469cae.

I think we need a install-test-modules or such that installs into the normal
directory.

Greetings,

Andres Freund

On 2023-03-07 Tu 14:37, Andres Freund wrote:

Hi,

On 2023-03-01 13:32:58 -0800, Andres Freund wrote:

On 2023-03-01 16:21:32 -0500, Andrew Dunstan wrote:

Perhaps the latest version will be more to your taste.

I'll check it out.

A simple conversion from an existing config failed with:
Can't use an undefined value as an ARRAY reference at /home/bf/src/pgbuildfarm-client-meson/PGBuild/Modules/TestICU.pm line 37.

I disabled TestICU and was able to progress past that.

Pushed a fix for that.

...
piculet-meson:HEAD [19:12:48] setting up db cluster (C)...
piculet-meson:HEAD [19:12:48] starting db (C)...
piculet-meson:HEAD [19:12:48] running installcheck (C)...
piculet-meson:HEAD [19:12:57] restarting db (C)...
piculet-meson:HEAD [19:12:59] running meson misc installchecks (C) ...
Branch: HEAD
Stage delay_executionInstallCheck-C failed with status 1

The failures are like this:

+ERROR:  extension "dummy_index_am" is not available
+DETAIL:  Could not open extension control file "/home/bf/bf-build/piculet-meson/HEAD/inst/share/postgresql/extension/dummy_index_am.control": No such file or directory.
+HINT:  The extension must first be installed on the system where PostgreSQL is running.

I assume this is in an interaction with b6a0d469cae.

I think we need a install-test-modules or such that installs into the normal
directory.

Exactly.

cheers

andrew

--

Andrew Dunstan
EDB:https://www.enterprisedb.com

Hi,

On 2023-03-07 15:47:54 -0500, Andrew Dunstan wrote:

On 2023-03-07 Tu 14:37, Andres Freund wrote:

The failures are like this:

+ERROR:  extension "dummy_index_am" is not available
+DETAIL:  Could not open extension control file "/home/bf/bf-build/piculet-meson/HEAD/inst/share/postgresql/extension/dummy_index_am.control": No such file or directory.
+HINT:  The extension must first be installed on the system where PostgreSQL is running.

I assume this is in an interaction with b6a0d469cae.

I think we need a install-test-modules or such that installs into the normal
directory.

Exactly.

Here's a prototype for that.

It adds an install-test-files target, Because we want to install into a normal
directory, I removed the necessary munging of the target paths from
meson.build and moved it into install-test-files. I also added DESTDIR
support, so that installing can redirect the directory if desired. That's used
for the tmp_install/ installation now.

I didn't like the number of arguments necessary for install_test_files, so I
changed it to use

--install target list of files

which makes it easier to use for further directories, if/when we need them.

Greetings,

Andres Freund

Hi,

On 2023-02-23 06:27:23 -0500, Andrew Dunstan wrote:

Yeah. For touch I think we can probably just get rid of this line in the
root meson.build:

touch = find_program('touch', native: true)

Yep.

For cp there doesn't seem to be a formal requirement, but there is a recipe
in src/common/unicode/meson.build that uses it, maybe that's what caused the
failure. On Windows/msvc we could just use copy instead, I think.

I don't know about using copy, it's very easy to get into trouble due to
interpreting forward slashes as options etc. I propose that for now we just
don't support update-unicode if cp isn't available - just as already not
available when wget isn't available.

Planning to apply something like the attached soon, unless somebody opposes
that plan.

Other unix tools we have a hard requirement on right now:
- sed - would be pretty easy to replace with something else
- tar, gzip - just for tests

I'm not sure it's worth working on not requiring those.

There's also flex, bison, perl, but those will stay a hard requirement for a
while longer... :)

Greetings,

Andres Freund

On 2023-03-07 18:26:21 -0800, Andres Freund wrote:

On 2023-02-23 06:27:23 -0500, Andrew Dunstan wrote:

Yeah. For touch I think we can probably just get rid of this line in the
root meson.build:

touch = find_program('touch', native: true)

Yep.

For cp there doesn't seem to be a formal requirement, but there is a recipe
in src/common/unicode/meson.build that uses it, maybe that's what caused the
failure. On Windows/msvc we could just use copy instead, I think.

I don't know about using copy, it's very easy to get into trouble due to
interpreting forward slashes as options etc. I propose that for now we just
don't support update-unicode if cp isn't available - just as already not
available when wget isn't available.

Planning to apply something like the attached soon, unless somebody opposes
that plan.

Done.

On 2023-03-07 Tu 20:29, Andres Freund wrote:

Hi,

On 2023-03-07 15:47:54 -0500, Andrew Dunstan wrote:

On 2023-03-07 Tu 14:37, Andres Freund wrote:

The failures are like this:

+ERROR:  extension "dummy_index_am" is not available
+DETAIL:  Could not open extension control file "/home/bf/bf-build/piculet-meson/HEAD/inst/share/postgresql/extension/dummy_index_am.control": No such file or directory.
+HINT:  The extension must first be installed on the system where PostgreSQL is running.

I assume this is in an interaction with b6a0d469cae.

I think we need a install-test-modules or such that installs into the normal
directory.

Exactly.

Here's a prototype for that.

It adds an install-test-files target, Because we want to install into a normal
directory, I removed the necessary munging of the target paths from
meson.build and moved it into install-test-files. I also added DESTDIR
support, so that installing can redirect the directory if desired. That's used
for the tmp_install/ installation now.

I didn't like the number of arguments necessary for install_test_files, so I
changed it to use

--install target list of files

which makes it easier to use for further directories, if/when we need them.

So if I understand this right, the way to use this would be something like:

    local $ENV{DESTDIR} = $installdir;

    run_log("meson compile -C $pgsql install-test-files");

Is that right? I did that but it didn't work :-(

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com