buildfarm does not test "make check"

On 08/13/2015 02:11 PM, Alvaro Herrera wrote:

So I added the brin isolation test back. Because it needs the
pageinspect module, it can only work under "make check", not
installcheck. The problem is that this means buildfarm will not run it,
because it only runs installcheck :-(

I realized that the important modules that run under "make check"
already have custom buildfarm modules, namely pg_upgrade and
test_decoding. I wonder if it would make more sense to have this be
customizable from the buildfarm server side, without having to resort to
writing a buildfarm module for each new thing. I envision a file hosted
in the buildfarm server, perhaps a JSON blob, that lists modules that
need "make check" treatment. Within the JSON object for each such
module there could live the details such as what files need to be
saved. That way, we could add more things to test without requiring the
buildfarm client to be upgraded each time.

This approach seems to have worked very well to customize the branches
that each animal builds, which is why I suggest that it be used here
too.

The buildfarm server doesn't control anything the clients do except it
provides them with a list of branches we are interested in. Even that is
invisible to the main program, and only used by the wrapper
run_branches.pl. The main program can run entirely offline and I'm going
to resist moves to make it otherwise. (Yes it needs access to git, but
even that can be finessed.)

If you want some way of specifying this, put it in the source, e.g.
invent a directory src/test/buildfarm_support and put it there. That way
committers have automatic access to it, which they certainly don't to
the buildfarm server. I'd rather just make it a small piece of perl
instead of JSON,though.

You're also going to have to handle the msvc side of things. That won't
be trivial. See discussion elsewhere today about how we've got that
wrong recently.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andrew Dunstan wrote:

The buildfarm server doesn't control anything the clients do except it
provides them with a list of branches we are interested in. Even that is
invisible to the main program, and only used by the wrapper run_branches.pl.
The main program can run entirely offline and I'm going to resist moves to
make it otherwise. (Yes it needs access to git, but even that can be
finessed.)

Understood.

If you want some way of specifying this, put it in the source, e.g. invent a
directory src/test/buildfarm_support and put it there. That way committers
have automatic access to it, which they certainly don't to the buildfarm
server. I'd rather just make it a small piece of perl instead of
JSON,though.

Makes plenty of sense, thanks.

You're also going to have to handle the msvc side of things. That won't be
trivial. See discussion elsewhere today about how we've got that wrong
recently.

Oh my. The pg_upgrade code in src/tools/msvc/vcregress.pl looks rather
unhealthy, and I don't see anything there to handle test_decoding, so I
assume that's done differently somehow. (For pg_upgrade surely we should
not be using a shell script ...)

I don't have time to go through this right now, but it's on my list of
things to think about.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 08/13/2015 02:52 PM, Alvaro Herrera wrote:

You're also going to have to handle the msvc side of things. That won't be
trivial. See discussion elsewhere today about how we've got that wrong
recently.

Oh my. The pg_upgrade code in src/tools/msvc/vcregress.pl looks rather
unhealthy, and I don't see anything there to handle test_decoding, so I
assume that's done differently somehow. (For pg_upgrade surely we should
not be using a shell script ...)

I don't have time to go through this right now, but it's on my list of
things to think about.

Don't worry about the existing cases. Just create something that handles
new test cases in a generic way.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Aug 13, 2015 at 2:11 PM, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

So I added the brin isolation test back. Because it needs the
pageinspect module, it can only work under "make check", not
installcheck. The problem is that this means buildfarm will not run it,
because it only runs installcheck :-(

Aren't "make installcheck" and "make check" supposed to test the same
set of things?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 08/14/2015 09:27 AM, Robert Haas wrote:

On Thu, Aug 13, 2015 at 2:11 PM, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

So I added the brin isolation test back. Because it needs the
pageinspect module, it can only work under "make check", not
installcheck. The problem is that this means buildfarm will not run it,
because it only runs installcheck :-(

Aren't "make installcheck" and "make check" supposed to test the same
set of things?

There are some cases that don't provide installcheck targets.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers