macOS - Sequoia CI task is stuck on the Postgres Github CI

Hi,

On 2025-11-26 16:44:00 +0300, Nazir Bilal Yavuz wrote:

I noticed that the macOS – Sequoia CI task is stuck on the PostgreSQL
GitHub CI [1], while the same task runs successfully on the CFBot [2].
On the CFBot, the worker jc-m2-1 is used, but the PostgreSQL GitHub CI
does not show a specific worker name although it reports
instance:PersistentWorkerInstance.

The PostgreSQL GitHub CI macOS tasks used to run on m4-dev, which I
believe was Andres’ worker machine. CC’ing Andres to check whether
something might be broken on the m4-dev worker.

The sequoia image was updated and my internet was a lot slower than usual for
some as of yet unknown reason. So it took a while to download the ~220GB of
image... It should be catching up now.

Greetings,

Andres Freund

On 2025-Nov-26, Nazir Bilal Yavuz wrote:

Hi,

I noticed that the macOS – Sequoia CI task is stuck on the PostgreSQL
GitHub CI [1], while the same task runs successfully on the CFBot [2].
On the CFBot, the worker jc-m2-1 is used, but the PostgreSQL GitHub CI
does not show a specific worker name although it reports
instance:PersistentWorkerInstance.

Maybe the problem is just that the macOS workers are too busy? If I go
to https://cirrus-ci.com/github/postgres/postgres/master I see a bunch
that are marked as "executing", and if you open those, you see that
they're all waiting until the "MacOS - Sequoia" task to complete, which
is state Scheduled. ... and while I was looking at one, it switched to
running.

(I only came here to report that buildfarm member Prion gets stuck in
one of the new test cases added by bc32a12e0db2. The culprit is
CATCACHE_FORCE_RELEASE. Not relevant to your investigation.)

--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
<Schwern> It does it in a really, really complicated way
<crab> why does it need to be complicated?
<Schwern> Because it's MakeMaker.

Hi,

On 2025-11-26 15:38:37 +0100, Álvaro Herrera wrote:

I noticed that the macOS – Sequoia CI task is stuck on the PostgreSQL
GitHub CI [1], while the same task runs successfully on the CFBot [2].
On the CFBot, the worker jc-m2-1 is used, but the PostgreSQL GitHub CI
does not show a specific worker name although it reports
instance:PersistentWorkerInstance.

Maybe the problem is just that the macOS workers are too busy? If I go
to https://cirrus-ci.com/github/postgres/postgres/master I see a bunch
that are marked as "executing", and if you open those, you see that
they're all waiting until the "MacOS - Sequoia" task to complete, which
is state Scheduled. ... and while I was looking at one, it switched to
running.

The runner was down trying to download a new version of the image, which was
slower than normal, which created a backlog. But now it's downloaded, and the
backlog should be processed in another ~20 minutes.

Separately: We do unfortunately have quite a few spurious failures
lately. NetBSD is the most common failing task, I think that really needs the
upgrade to IPC::Run we've been talking about... But there are some others too.

Greetings,

Andres Freund