regression.out and regression.diff

Vince Vielhaber writes:

is regression.out and/or regression.diff deleted if the tests pass? I've
never seen all tests pass so I don't know but I just had someone tell me
that it does.

We've come to a point where all tests should pass all the time on all
supported platforms. If they don't, that's a bug and should be posted to
a mailing list.

The test output should come from 'gmake check', not the test against an
already installed server ('installcheck').

--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/

On Tue, 27 Feb 2001, Peter Eisentraut wrote:

Vince Vielhaber writes:

is regression.out and/or regression.diff deleted if the tests pass? I've
never seen all tests pass so I don't know but I just had someone tell me
that it does.

We've come to a point where all tests should pass all the time on all
supported platforms. If they don't, that's a bug and should be posted to
a mailing list.

The test output should come from 'gmake check', not the test against an
already installed server ('installcheck').

It was a snapshot - no idea what kind of shape it was in and I don't
recall how I did the check, but it may have been installcheck.

Either way my question still needs to be answered. If there are no
errors, is regression.out and regression.diff deleted? I can see the
diff file being empty, but deleted?

Vince.
--
==========================================================================
Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net
128K ISDN from $22.00/mo - 56K Dialup from $16.00/mo at Pop4 Networking
Online Campground Directory http://www.camping-usa.com
Online Giftshop Superstore http://www.cloudninegifts.com
==========================================================================

On Tue, 27 Feb 2001, Vince Vielhaber wrote:

On Tue, 27 Feb 2001, Peter Eisentraut wrote:

Vince Vielhaber writes:

is regression.out and/or regression.diff deleted if the tests pass? I've
never seen all tests pass so I don't know but I just had someone tell me
that it does.

We've come to a point where all tests should pass all the time on all
supported platforms. If they don't, that's a bug and should be posted to
a mailing list.

The test output should come from 'gmake check', not the test against an
already installed server ('installcheck').

It was a snapshot - no idea what kind of shape it was in and I don't
recall how I did the check, but it may have been installcheck.

Make that "must have been installcheck" 'cuze I just reran it with
gmake check and it passed all but the ignored random test.

Vince.
--
==========================================================================
Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net
128K ISDN from $22.00/mo - 56K Dialup from $16.00/mo at Pop4 Networking
Online Campground Directory http://www.camping-usa.com
Online Giftshop Superstore http://www.cloudninegifts.com
==========================================================================

Vince Vielhaber writes:

It was a snapshot - no idea what kind of shape it was in and I don't
recall how I did the check, but it may have been installcheck.

Even snapshots should work all the time. If not, it should be reported,
not posted to a web page.

Either way my question still needs to be answered. If there are no
errors, is regression.out and regression.diff deleted? I can see the
diff file being empty, but deleted?

The diff file always exists; if all tests passed, it's empty. The .out
file always contains a copy of what you see on the screen (i.e., the
ok/FAIL). Which one of these two you want depends on what you are trying
to achieve, but the .out file is probably less necessary, because if there
are failed tests, you will see in the diff file.

--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/

Peter Eisentraut <peter_e@gmx.net> writes:

Either way my question still needs to be answered. If there are no
errors, is regression.out and regression.diff deleted? I can see the
diff file being empty, but deleted?

The diff file always exists; if all tests passed, it's empty.

Forgot your own code already, Peter?

if [ -s "$diff_file" ]; then
echo "The differences that caused some tests to fail can be viewed in the"
echo "file \`$diff_file'. A copy of the test summary that you see"
echo "above is saved in the file \`$result_summary_file'."
echo
else
rm -f "$diff_file" "$result_summary_file"
fi

regards, tom lane

Vince Vielhaber <vev@michvhf.com> writes:

The test output should come from 'gmake check', not the test against an
already installed server ('installcheck').

It was a snapshot - no idea what kind of shape it was in and I don't
recall how I did the check, but it may have been installcheck.

Make that "must have been installcheck" 'cuze I just reran it with
gmake check and it passed all but the ignored random test.

I have no idea why Peter thinks 'make installcheck' should be less
reliable than 'make check'. If installcheck fails for you, let's
see that too.

regards, tom lane

Tom Lane writes:

I have no idea why Peter thinks 'make installcheck' should be less
reliable than 'make check'. If installcheck fails for you, let's
see that too.

In the test run that Vince had posted to his web tool, the server process
apparently didn't have write permission to the source tree, so all the
tests that did a COPY failed, plus all subsequent tests that depended on
those tables. Additionally, the installcheck is also prone to fail if
template1 was initialized with a different multibyte encoding, if there
was a different locale during initdb, or if there's something fishy in
postgresql.conf. At least I wouldn't accept installcheck output as a
final result before seeing 'check'.

Additionally, make check also tests 'make install' and 'initdb'
robustness, which installcheck doesn't do, so the former should be
preferred as final test result.

--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/