[PATCH] Add GitLab CI to PostgreSQL

On 2023-07-04 Tu 19:44, Newhouse, Robin wrote:

Hello!

I propose the attached patch to be applied on the 'master' branch

of PostgreSQL to add GitLab CI automation alongside Cirrus CI in the
PostgreSQL repository.

It is not intended to be a replacement for Cirrus CI, but simply
suggestion for the

PostgreSQL project to host centrally a Gitlab CI definition for those
who prefer to use

it while developing/testing PostgreSQL.

The intent is to facilitate collaboration among GitLab users, promote
standardization

and consistency, and ultimately, improve testing and code quality.

This seems very RedHat-centric, which I'm not sure is a good idea. Also,
shouldn't at least some of these recipes call dnf and dnf-builddep
instead of yum and yum-build-dep?

If we're going to do this then arguably we should also be supporting
GitHub Actions and who knows what other CI frameworks. There is a case
for us special casing Cirrus CI because it's used for the cfbot.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

On Wed, 5 Jul 2023 at 15:22, Andrew Dunstan <andrew@dunslane.net> wrote:

On 2023-07-04 Tu 19:44, Newhouse, Robin wrote:

Hello!

I propose the attached patch to be applied on the 'master' branch
of PostgreSQL to add GitLab CI automation alongside Cirrus CI in the PostgreSQL repository.

Can you configure GitLab to use a .gitlab-ci.yml file that is not in
the root directory?

It is not intended to be a replacement for Cirrus CI, but simply suggestion for the
PostgreSQL project to host centrally a Gitlab CI definition for those who prefer to use
it while developing/testing PostgreSQL.

The intent is to facilitate collaboration among GitLab users, promote standardization
and consistency, and ultimately, improve testing and code quality.

This seems very RedHat-centric, which I'm not sure is a good idea. Also, shouldn't at least some of these recipes call dnf and dnf-builddep instead of yum and yum-build-dep?

I don't think it's bad to add an automated test suite for redhat-based images?

If we're going to do this then arguably we should also be supporting GitHub Actions and who knows what other CI frameworks. There is a case for us special casing Cirrus CI because it's used for the cfbot.

I think there's a good case for _not_ using Cirrus CI, namely that the
license may be prohibitive, self-management of the build system
(storage of artifacts, UI, database) is missing for Cirrus CI, and it
also seems to be unable to run automated CI on repositories that
aren't hosted on Github.
I think these are good arguments for adding a GitLab CI configuration.

Unless the cfbot is entirely under management of the PostgreSQL
project (which it doesn't appear to be, as far as I know the URL is
still cfbot.cputube.org indicating some amount of external control)
the only special casing for Cirrus CI within the project seems to be
"people have experience with this tool", which is good, but not
exclusive to Cirrus CI - clearly there are also people here who have
experience with (or are interested in) maintaining GitLab CI
configurations.

Kind regards,

Matthias van de Meent
Neon (https://neon.tech)

On 2023-07-05 We 11:58, Matthias van de Meent wrote:

On Wed, 5 Jul 2023 at 15:22, Andrew Dunstan<andrew@dunslane.net> wrote:

On 2023-07-04 Tu 19:44, Newhouse, Robin wrote:

Hello!

I propose the attached patch to be applied on the 'master' branch
of PostgreSQL to add GitLab CI automation alongside Cirrus CI in the PostgreSQL repository.

Can you configure GitLab to use a .gitlab-ci.yml file that is not in
the root directory?

It is not intended to be a replacement for Cirrus CI, but simply suggestion for the
PostgreSQL project to host centrally a Gitlab CI definition for those who prefer to use
it while developing/testing PostgreSQL.

The intent is to facilitate collaboration among GitLab users, promote standardization
and consistency, and ultimately, improve testing and code quality.

This seems very RedHat-centric, which I'm not sure is a good idea. Also, shouldn't at least some of these recipes call dnf and dnf-builddep instead of yum and yum-build-dep?

I don't think it's bad to add an automated test suite for redhat-based images?

I didn't suggest it wasn't just that the coverage should be broader.

If we're going to do this then arguably we should also be supporting GitHub Actions and who knows what other CI frameworks. There is a case for us special casing Cirrus CI because it's used for the cfbot.

I think there's a good case for _not_ using Cirrus CI, namely that the
license may be prohibitive, self-management of the build system
(storage of artifacts, UI, database) is missing for Cirrus CI, and it
also seems to be unable to run automated CI on repositories that
aren't hosted on Github.
I think these are good arguments for adding a GitLab CI configuration.

Unless the cfbot is entirely under management of the PostgreSQL
project (which it doesn't appear to be, as far as I know the URL is
still cfbot.cputube.org indicating some amount of external control)
the only special casing for Cirrus CI within the project seems to be
"people have experience with this tool", which is good, but not
exclusive to Cirrus CI - clearly there are also people here who have
experience with (or are interested in) maintaining GitLab CI
configurations.

I would not assume too much from the URL. The PostgreSQL BuildFarm
operated for many years with a URL that was not under postgresql.org. I
assume the URL is partly a function of the fact that Thomas started the
cfbot as a bit of a skunkworks project. However it's run, the fact is
that the project relies to some extent on it.

cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

On 06.07.23 13:32, Andrew Dunstan wrote:

This seems very RedHat-centric, which I'm not sure is a good idea. Also, shouldn't at least some of these recipes call dnf and dnf-builddep instead of yum and yum-build-dep?

I don't think it's bad to add an automated test suite for redhat-based images?

I didn't suggest it wasn't just that the coverage should be broader.

If we were to accept this (or other providers besides Cirrus), then I
think they should run the exact same configurations that we have for
Cirrus right now (or possibly subsets or supersets, depending on
availability and capabilities). Those have been put there with a lot of
care to get efficient and reasonably broad coverage. There is no point
in starting that whole journey over again.

If someone thinks we should have more coverage for Red Hat-based
platforms, then let's put that into the Cirrus configuration. That
should be independent of the choice of CI provider.

On Wed, Jul 5, 2023 at 6:22 AM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2023-07-04 Tu 19:44, Newhouse, Robin wrote:

Hello!

I propose the attached patch to be applied on the 'master' branch

of PostgreSQL to add GitLab CI automation alongside Cirrus CI in the PostgreSQL repository.

It is not intended to be a replacement for Cirrus CI, but simply suggestion for the

PostgreSQL project to host centrally a Gitlab CI definition for those who prefer to use

it while developing/testing PostgreSQL.

The intent is to facilitate collaboration among GitLab users, promote standardization

and consistency, and ultimately, improve testing and code quality.

This seems very RedHat-centric, which I'm not sure is a good idea.

A few years ago, a proposal to use CentOS may not have been a bad
proposal. But since Redhat changed CentOS to be an upstream distro
(rather than a rebuild of RHEL), I do see a reason to consider RHEL as
a candidate in our CI.

I think the idea of a pre-buildfarm CI is to enable contributors catch
problems before they're merged, or even before proposed as a patch to
the community. So if our CI includes support for a prominent Linux
distro, I think it's worth it, to provide coverage for the large
ecosystem that's based on RHEL and its derivatives.

Would using RockyLinux assuage these concerns? Perhaps, it would.

If we're going to do this then arguably we should also be supporting GitHub Actions and who knows what other CI frameworks. There is a case for us special casing Cirrus CI because it's used for the cfbot.

We've already lost that battle by supporting one
commercial/non-community provider. From Anrdres' email [1]/messages/by-id/20211001222752.wrz7erzh4cajvgp6@alap3.anarazel.de:

An obvious criticism of the effort to put CI runner infrastructure into core
is that they are effectively all proprietary technology, and that we should be
hesistant to depend too much on one of them. I think that's a valid
concern. However, once one CI integration is done, a good chunk (but not all!)
the work is transferrable to another CI solution, which I do think reduces the
dependency sufficiently.

So it seems that supporting more than one CI was always on the cards.
Cirrus was chosen for its advantages that Andres mentions in the
email, but also for the reason that cfbot had chosen Cirrus. I can
imagine if cfbot was developed against some other CI, it's very likely
that we'd be using that other CI instead of Cirrus.

[1]: /messages/by-id/20211001222752.wrz7erzh4cajvgp6@alap3.anarazel.de

Best regards,
Gurjeet
http://Gurje.et

On 6 Jul 2023, at 20:10, Gurjeet Singh <gurjeet@singh.im> wrote:

I can
imagine if cfbot was developed against some other CI, it's very likely
that we'd be using that other CI instead of Cirrus.

The CFBot originally used Travis, but switched in late 2020 when Travis almost
over night become hard to use for open source projects:

https://github.com/macdice/cfbot/commit/a62aa6d77dd4cc7f0a5549db378cd6f1cf25c0e2

These systems come and go, and each have their quirks. Having options is good,
but maintaining multiple ones isn't necessarily a free fire-and-forget type of
thing for the community.

--
Daniel Gustafsson

Hi,

On 2023-07-04 23:44:45 +0000, Newhouse, Robin wrote:

I propose the attached patch to be applied on the 'master' branch
of PostgreSQL to add GitLab CI automation alongside Cirrus CI in the PostgreSQL repository.

It is not intended to be a replacement for Cirrus CI, but simply suggestion for the
PostgreSQL project to host centrally a Gitlab CI definition for those who prefer to use
it while developing/testing PostgreSQL.

One way to avoid duplicated CI definition could be to use for gitlab-ci to use
cirrus-cli to run the cirrus CI tests within gitlab ci.

Realistically I think adding a separate CI definition would entail committers
needing to run that CI at least occasionally. If we make the different CI envs
more similar, that becomes less of a necessity.

+default:
+  # Base image for builds and tests unless otherwise defined
+  image: fedora:latest
+  # Extend build jobs to have longer timeout as the default GitLab
+  # timeout (1h) is often not enough
+  timeout: 3h

IMO we shouldn't add CI that doesn't complete within well under an hour, it's
too expensive workflow wise.

+fedora:
+  stage: build
+  variables:
+    GIT_STRATEGY: fetch
+    GIT_SUBMODULE_STRATEGY: normal
+  script:
+    # Install dependencies
+    - yum install -y yum-utils perl
+    - yum-builddep -y postgresql
+    - *build-postgres-def

My experience is that installing dependencies on each run is way too slow to
be practical. I also found that it often causes temporary failures due to
network issues etc. For cirrus-ci we create VM and docker images on a regular
schedule (three times a week right now) - if there's interest in building
fedora containers that'd be easy.

I'd be open to switching one of the cirrus-CI tasks over to fedora, fwiw.

+# From https://github.com/postgres/postgres/blob/master/.cirrus.yml
+.create-user: &create-user-def
+    - useradd -m postgres
+    - chown -R postgres:postgres .
+    - mkdir -p ${CCACHE_DIR}
+    - chown -R postgres:postgres ${CCACHE_DIR}
+    - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf
+    - su postgres -c "ulimit -l -H && ulimit -l -S"
+    # Can't change container's kernel.core_pattern. Postgres user can't write
+    # to / normally. Change that.
+    - chown root:postgres /
+    - chmod g+rwx /

If we need duplicated stanzas like this, we should instead move them out into
scripts that we can use from different CI environments.

+# Similar to https://github.com/postgres/postgres/blob/master/.cirrus.yml
+fedora meson:
+  stage: build
+  variables:
+    GIT_STRATEGY: fetch
+    GIT_SUBMODULE_STRATEGY: normal
+  script:
+    # Meson system only exists on master branch currently

Master and 16 now...

+    - if [ ! -f meson.build ]; then exit 0; fi
+    # Install dependencies
+    - yum install -y yum-utils perl perl-IPC-Run meson ninja-build
+    - yum-builddep -y postgresql
+    # Create postgres user
+    - *create-user-def
+    # Configure
+    - su postgres -c 'meson setup --buildtype=debug --auto-features=disabled -Dtap_tests=enabled build'
+    # Build
+    - su postgres -c 'ninja -C build -j 2'
+    # Minimal test
+    - su postgres -c 'meson test $MTEST_ARGS --num-processes 2 tmp_install cube/regress pg_ctl/001_start_stop'
+    # Run all tests
+    - su postgres -c 'meson test $MTEST_ARGS --num-processes 2'
+  artifacts:
+    when: always  # Must be able to see logs
+    paths:
+      - build/meson-logs/testlog.txt

FWIW, that's not enough to be able to debug problems. You really also need the
log files created by failing tests.

Greetings,

Andres Freund

On Thu, Jul 6, 2023 at 11:27 AM Daniel Gustafsson <daniel@yesql.se> wrote:

On 6 Jul 2023, at 20:10, Gurjeet Singh <gurjeet@singh.im> wrote:

I can
imagine if cfbot was developed against some other CI, it's very likely
that we'd be using that other CI instead of Cirrus.

The CFBot originally used Travis, but switched in late 2020 when Travis almost
over night become hard to use for open source projects:

https://github.com/macdice/cfbot/commit/a62aa6d77dd4cc7f0a5549db378cd6f1cf25c0e2

Thanks for providing the historical context! A for-profit entity,
despite their best intentions, and sometimes by no fault of their own,
may not survive. It's not that a non-profits are guaranteed to
survive, but the conditions they operate in are drastically different
than those of for-profit ones.

These systems come and go, and each have their quirks.

I'm sure the community has seen enough of such disappearances over the
years, which is why I was surprised to see the adoption of Cirrus in
core (after I had stopped paying attention to Postgres hackers list
for a few years). Having read that whole discussion, though, I do see
the immense value Cirrus CI provides.

Having options is good,
but maintaining multiple ones isn't necessarily a free fire-and-forget type of
thing for the community.

By not adopting at least one other CI, it'd seem like the community
is favoring Cirrus over others; and that doesn't feel good.

Best regards,
Gurjeet
http://Gurje.et