Running TAP regression tests under windows/msvc

Started by Mark Dilgerabout 9 years ago3 messagesgeneral
Jump to latest
#1Mark Dilger
mark.dilger@enterprisedb.com

Hello,

I am attempting to get the tap tests working under windows so as to
help review patches for the 10.0 development cycle. I can compile
the sources on windows 2008 using the MS Visual C and run the

vcregress.bat check

without any problems or failures. In an attempt to run the tap tests,
I have edited config_default.pl as follows:

diff --git a/src/tools/msvc/config_default.pl b/src/tools/msvc/config_default.pl
index 97f1af8..1e7b19d 100644
--- a/src/tools/msvc/config_default.pl
+++ b/src/tools/msvc/config_default.pl
@@ -16,7 +16,7 @@ our $config = {
        extraver  => undef,    # --with-extra-version=<string>
        gss       => undef,    # --with-gssapi=<path>
        nls       => undef,    # --enable-nls=<path>
-       tap_tests => undef,    # --enable-tap-tests
+       tap_tests => 1,        # --enable-tap-tests
        tcl       => undef,    # --with-tls=<path>
        perl      => undef,    # --with-perl
        python    => undef,    # --with-python=<path>

and when I run

vcregress.bat bincheck

I get a few failures. Am I doing something wrong, or are these failures
the same for other folks? A portion of the log of the regressions follows:

All tests successful.
Files=5, Tests=17, 323 wallclock secs ( 0.08 usr + 0.03 sys = 0.11 CPU)
Result: PASS
t/001_pgbench.pl ..
1..3
ok 1 - concurrent OID generation: exit code 0
ok 2 - concurrent OID generation: no stderr
ok 3 - concurrent OID generation: matches
ok
All tests successful.
Files=1, Tests=3, 12 wallclock secs ( 0.08 usr + 0.03 sys = 0.11 CPU)
Result: PASS

# Failed test 'SQL CLUSTER run: SQL found in server log'
# at c:/jenkins/workspace/unicorns/postgresql/src/test/perl/PostgresNode.pm line 1321.
# ''
# doesn't match '(?^:statement: CLUSTER;)'

# Failed test 'cluster specific table: SQL found in server log'
# at c:/jenkins/workspace/unicorns/postgresql/src/test/perl/PostgresNode.pm line 1321.
# ''
# doesn't match '(?^:statement: CLUSTER test1;)'
# Looks like you failed 2 tests of 14.
t/010_clusterdb.pl ........
1..14
ok 1 - clusterdb --help exit code 0
ok 2 - clusterdb --help goes to stdout
ok 3 - clusterdb --help nothing to stderr
ok 4 - clusterdb --version exit code 0
ok 5 - clusterdb --version goes to stdout
ok 6 - clusterdb --version nothing to stderr
ok 7 - clusterdb with invalid option nonzero exit code
ok 8 - clusterdb with invalid option prints error message
ok 9 - clusterdb exit code 0
not ok 10 - SQL CLUSTER run: SQL found in server log
ok 11 - fails with nonexistent table
ok 12 - clusterdb -t test1 exit code 0
not ok 13 - cluster specific table: SQL found in server log
ok 14 - clusterdb with connection string
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/14 subtests

# Failed test 'cluster all databases: SQL found in server log'
# at c:/jenkins/workspace/unicorns/postgresql/src/test/perl/PostgresNode.pm line 1321.
# ''
# doesn't match '(?^s:statement: CLUSTER.*statement: CLUSTER)'
# Looks like you failed 1 test of 2.
t/011_clusterdb_all.pl ....
1..2
ok 1 - clusterdb -a exit code 0
not ok 2 - cluster all databases: SQL found in server log
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests

# Failed test 'SQL CREATE DATABASE run: SQL found in server log'
# at c:/jenkins/workspace/unicorns/postgresql/src/test/perl/PostgresNode.pm line 1321.
# ''
# doesn't match '(?^:statement: CREATE DATABASE foobar1)'

# Failed test 'create database with encoding: SQL found in server log'
# at c:/jenkins/workspace/unicorns/postgresql/src/test/perl/PostgresNode.pm line 1321.
# ''
# doesn't match '(?^:statement: CREATE DATABASE foobar2 ENCODING 'LATIN1')'
# Looks like you failed 2 tests of 13.
t/020_createdb.pl .........
1..13
ok 1 - createdb --help exit code 0
ok 2 - createdb --help goes to stdout
ok 3 - createdb --help nothing to stderr
ok 4 - createdb --version exit code 0
ok 5 - createdb --version goes to stdout
ok 6 - createdb --version nothing to stderr
ok 7 - createdb with invalid option nonzero exit code
ok 8 - createdb with invalid option prints error message
ok 9 - createdb foobar1 exit code 0
not ok 10 - SQL CREATE DATABASE run: SQL found in server log
ok 11 - createdb -l C -E LATIN1 -T template0 foobar2 exit code 0
not ok 12 - create database with encoding: SQL found in server log
ok 13 - fails if database already exists
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/13 subtests

# Failed test 'SQL CREATE EXTENSION run: SQL found in server log'
# at c:/jenkins/workspace/unicorns/postgresql/src/test/perl/PostgresNode.pm line 1321.
# ''
# doesn't match '(?^:statement: CREATE EXTENSION "plpgsql")'
# Looks like you failed 1 test of 14.

Thanks in advance for any clarification regarding what I might be doing wrong.

Mark Dilger

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#2Mark Dilger
mark.dilger@enterprisedb.com
In reply to: Mark Dilger (#1)
Re: Running TAP regression tests under windows/msvc

On Mar 7, 2017, at 12:24 PM, Mark Dilger <hornschnorter@gmail.com> wrote:

Hello,

I am attempting to get the tap tests working under windows so as to
help review patches for the 10.0 development cycle. I can compile
the sources on windows 2008 using the MS Visual C and run the

vcregress.bat check

without any problems or failures. In an attempt to run the tap tests,
I have edited config_default.pl as follows:

diff --git a/src/tools/msvc/config_default.pl b/src/tools/msvc/config_default.pl
index 97f1af8..1e7b19d 100644
--- a/src/tools/msvc/config_default.pl
+++ b/src/tools/msvc/config_default.pl
@@ -16,7 +16,7 @@ our $config = {
extraver  => undef,    # --with-extra-version=<string>
gss       => undef,    # --with-gssapi=<path>
nls       => undef,    # --enable-nls=<path>
-       tap_tests => undef,    # --enable-tap-tests
+       tap_tests => 1,        # --enable-tap-tests
tcl       => undef,    # --with-tls=<path>
perl      => undef,    # --with-perl
python    => undef,    # --with-python=<path>

and when I run

vcregress.bat bincheck

I get a few failures. Am I doing something wrong, or are these failures
the same for other folks? A portion of the log of the regressions follows:

I added a bit of debugging logic to PostgresNode.pm, to print out the
name of the log file being grep'd and the size of that file, and it seems
in these cases the log file is of size zero (empty).

not ok 10 - SQL CLUSTER run: SQL found in server log c:/jenkins/workspace/unicorns/postgresql/src/bin/scripts/tmp_check/log/010_clusterdb_main.log, length 0
ok 11 - fails with nonexistent table
ok 12 - clusterdb -t test1 exit code 0
not ok 13 - cluster specific table: SQL found in server log c:/jenkins/workspace/unicorns/postgresql/src/bin/scripts/tmp_check/log/010_clusterdb_main.log, length 0
ok 14 - clusterdb with connection string
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/14 subtests

Perhaps there is a race condition between when the test is run and when the
log file is flushed? I'm just guessing here....

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#3Michael Paquier
michael@paquier.xyz
In reply to: Mark Dilger (#2)
Re: Running TAP regression tests under windows/msvc

On Wed, Mar 8, 2017 at 6:30 AM, Mark Dilger <hornschnorter@gmail.com> wrote:

On Mar 7, 2017, at 12:24 PM, Mark Dilger <hornschnorter@gmail.com> wrote:

Hello,

I am attempting to get the tap tests working under windows so as to
help review patches for the 10.0 development cycle. I can compile
the sources on windows 2008 using the MS Visual C and run the

vcregress.bat check

without any problems or failures. In an attempt to run the tap tests,
I have edited config_default.pl as follows:

diff --git a/src/tools/msvc/config_default.pl b/src/tools/msvc/config_default.pl
index 97f1af8..1e7b19d 100644
--- a/src/tools/msvc/config_default.pl
+++ b/src/tools/msvc/config_default.pl
@@ -16,7 +16,7 @@ our $config = {
extraver  => undef,    # --with-extra-version=<string>
gss       => undef,    # --with-gssapi=<path>
nls       => undef,    # --enable-nls=<path>
-       tap_tests => undef,    # --enable-tap-tests
+       tap_tests => 1,        # --enable-tap-tests
tcl       => undef,    # --with-tls=<path>
perl      => undef,    # --with-perl
python    => undef,    # --with-python=<path>

and when I run

vcregress.bat bincheck

I get a few failures. Am I doing something wrong, or are these failures
the same for other folks? A portion of the log of the regressions follows:

I don't think you are doing anything wrong here.

I added a bit of debugging logic to PostgresNode.pm, to print out the
name of the log file being grep'd and the size of that file, and it seems
in these cases the log file is of size zero (empty).

not ok 10 - SQL CLUSTER run: SQL found in server log c:/jenkins/workspace/unicorns/postgresql/src/bin/scripts/tmp_check/log/010_clusterdb_main.log, length 0
ok 11 - fails with nonexistent table
ok 12 - clusterdb -t test1 exit code 0
not ok 13 - cluster specific table: SQL found in server log c:/jenkins/workspace/unicorns/postgresql/src/bin/scripts/tmp_check/log/010_clusterdb_main.log, length 0
ok 14 - clusterdb with connection string
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/14 subtests

Perhaps there is a race condition between when the test is run and when the
log file is flushed? I'm just guessing here....

Just running now the tests by myself to see if I could reproduce your
failures, I have bumped before that into failures for the tests of
pg_ctl and the tests of pg_rewind not responding.. This needs some
serious investigation.

The buildfarm is not running those tests by the way, that's annoying.
--
Michael

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general