ci: reduce macos test concurrency

Started by Andres Freundover 3 years ago2 messageshackers
Jump to latest
#1Andres Freund
andres@anarazel.de

Hi,

The x86 mac VMs from cirrus-ci claim to have 12 CPUs, but when working on
initdb caching for tests I noticed that using all those CPUs for tests hurts
the test times noticably.

See [1]https://cirrus-ci.com/build/5254074546257920 (note that the overall time is influenced by different degrees of
cache hit ratios):

concurrency test time:
4 05:58
6 05:09
8 04:58
10 05:58
12 (current) 06:58

There's a fair bit of run-to-run variance, but the rough shape of these
looks repeatable.

I suspect the VMs might be overcommitted a fair bit - or macos just scales
poorly. Cirrus-ci apparently is switching to M1 based macs, which could be
related. It'd be good for us do that switch, as it'd give use ARM coverage for
CI / cfbot. See also [3]/messages/by-id/CAN55FZ2R+XufuVgJ8ew_yDBk48PgXEBvyKNvnNdTTVyczbQj0g@mail.gmail.com.

In 15 (and thus autoconf) the timings differ a bit less [2]https://cirrus-ci.com/task/4888800445857792:

concurrency test time:
4 06:54
6 05:43
8 06:09
10 06:01
12 (current) 06:38

Looks like changing TEST_JOBS=6 or 8 would be a good idea.

Greetings,

Andres Freund

[1]: https://cirrus-ci.com/build/5254074546257920
[2]: https://cirrus-ci.com/task/4888800445857792
[3]: /messages/by-id/CAN55FZ2R+XufuVgJ8ew_yDBk48PgXEBvyKNvnNdTTVyczbQj0g@mail.gmail.com

#2Andres Freund
andres@anarazel.de
In reply to: Andres Freund (#1)
Re: ci: reduce macos test concurrency

Hi,

On 2022-09-26 21:02:08 -0700, Andres Freund wrote:

The x86 mac VMs from cirrus-ci claim to have 12 CPUs, but when working on
initdb caching for tests I noticed that using all those CPUs for tests hurts
the test times noticably.

See [1] (note that the overall time is influenced by different degrees of
cache hit ratios):

concurrency test time:
4 05:58
6 05:09
8 04:58
10 05:58
12 (current) 06:58

There's a fair bit of run-to-run variance, but the rough shape of these
looks repeatable.

I suspect the VMs might be overcommitted a fair bit - or macos just scales
poorly. Cirrus-ci apparently is switching to M1 based macs, which could be
related. It'd be good for us do that switch, as it'd give use ARM coverage for
CI / cfbot. See also [3].

In 15 (and thus autoconf) the timings differ a bit less [2]:

concurrency test time:
4 06:54
6 05:43
8 06:09
10 06:01
12 (current) 06:38

Looks like changing TEST_JOBS=6 or 8 would be a good idea.

Set it to 8 now.

Greetings,

Andres Freund