convert libpq uri-regress tests to tap test
Hi,
When verifying that the meson port actually runs all perl based tests I came
across src/interfaces/libpq/test/regress.pl. Instead of running tests yet
another way, it seems better to convert it to a tap test.
I hope others agree?
Where would we want that test to live? Right now we have the slightly odd
convention that some tap tests live in src/test/{misc,modules,...}. But
e.g. frontend binary ones are below src/bin/.
For now I've left it in src/interfaces/libpq/test, with the test in
t/001_uri.pl. But we should at least get rid of the test/...
Perhaps we should just rename src/test/modules/libpq_pipeline to
src/test/modules/libpq and move uri-regress in there as well?
The tap test needs a bit more polish, mostly posted this to get some feedback
before wasting more time :)
Greetings,
Andres Freund
Attachments:
v1-0001-Convert-src-interfaces-libpq-test-to-a-tap-test.patchtext/x-diff; charset=us-asciiDownload+274-305
On 2/23/22 15:30, Andres Freund wrote:
Hi,
When verifying that the meson port actually runs all perl based tests I came
across src/interfaces/libpq/test/regress.pl. Instead of running tests yet
another way, it seems better to convert it to a tap test.I hope others agree?
yes
Where would we want that test to live? Right now we have the slightly odd
convention that some tap tests live in src/test/{misc,modules,...}. But
e.g. frontend binary ones are below src/bin/.For now I've left it in src/interfaces/libpq/test, with the test in
t/001_uri.pl. But we should at least get rid of the test/...Perhaps we should just rename src/test/modules/libpq_pipeline to
src/test/modules/libpq and move uri-regress in there as well?
WFM
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On 23 Feb 2022, at 21:30, Andres Freund <andres@anarazel.de> wrote:
When verifying that the meson port actually runs all perl based tests I came
across src/interfaces/libpq/test/regress.pl. Instead of running tests yet
another way, it seems better to convert it to a tap test.I hope others agree?
Many +1's on that.
The tap test needs a bit more polish, mostly posted this to get some feedback
before wasting more time :)
Skimming the new test it seems structurally fine to me.
--
Daniel Gustafsson https://vmware.com/
Andrew Dunstan <andrew@dunslane.net> writes:
On 2/23/22 15:30, Andres Freund wrote:
Perhaps we should just rename src/test/modules/libpq_pipeline to
src/test/modules/libpq and move uri-regress in there as well?
WFM
+1
regards, tom lane
Hi,
On 2022-02-23 15:57:08 -0500, Tom Lane wrote:
Andrew Dunstan <andrew@dunslane.net> writes:
On 2/23/22 15:30, Andres Freund wrote:
Perhaps we should just rename src/test/modules/libpq_pipeline to
src/test/modules/libpq and move uri-regress in there as well?WFM
+1
Cool.
One question on making that happen: Right now the makefiles building C stuff
in src/test/modules all have the contrib-style stanza to build via pgxs. But
afaics pgxs doesn't support building multiple programs. Which
src/test/modules/libpq would need to.
Aaics there's currently no way to have Mkvcbuild.pm build multiple programs in
one dir via its makefile parsing stuff? Andrew, any suggestions for not
needing to spell this out in both the makefile and Mkvcbuild.pm?
Separately: I don't really understand why we do the whole if USE_PGXS dance in
src/test/modules?
- Andres
On 2022-Feb-23, Andres Freund wrote:
When verifying that the meson port actually runs all perl based tests I came
across src/interfaces/libpq/test/regress.pl. Instead of running tests yet
another way, it seems better to convert it to a tap test.I hope others agree?
WFM.
Perhaps we should just rename src/test/modules/libpq_pipeline to
src/test/modules/libpq and move uri-regress in there as well?
Well, building multiple binaries would require some trickery that might
be excessive for this little tool. But +1 to that on principle.
--
Álvaro Herrera Valdivia, Chile — https://www.EnterpriseDB.com/
"La virtud es el justo medio entre dos defectos" (Aristóteles)
On 2022-Feb-23, Andres Freund wrote:
Separately: I don't really understand why we do the whole if USE_PGXS dance in
src/test/modules?
Yeah, it seems a bit silly. I'm not opposed to making these makefiles
unconditionally do the PGXS thing -- or the non-PGXS thing, if that
makes it easier to build multiple binaries.
--
Álvaro Herrera Valdivia, Chile — https://www.EnterpriseDB.com/
"La grandeza es una experiencia transitoria. Nunca es consistente.
Depende en gran parte de la imaginación humana creadora de mitos"
(Irulan)
On 2/23/22 16:16, Andres Freund wrote:
Hi,
On 2022-02-23 15:57:08 -0500, Tom Lane wrote:
Andrew Dunstan <andrew@dunslane.net> writes:
On 2/23/22 15:30, Andres Freund wrote:
Perhaps we should just rename src/test/modules/libpq_pipeline to
src/test/modules/libpq and move uri-regress in there as well?WFM
+1
Cool.
One question on making that happen: Right now the makefiles building C stuff
in src/test/modules all have the contrib-style stanza to build via pgxs. But
afaics pgxs doesn't support building multiple programs. Which
src/test/modules/libpq would need to.
That's my understanding also.
Aaics there's currently no way to have Mkvcbuild.pm build multiple programs in
one dir via its makefile parsing stuff? Andrew, any suggestions for not
needing to spell this out in both the makefile and Mkvcbuild.pm?
Well, it should be a SMOC. If you can solve the first problem I'm sure I
can come up with a solution for Mkvcbuild.pm. But until we see the shape
of the pgxs changes, planning for Mkcvbuild.pm changes seems remature.
Separately: I don't really understand why we do the whole if USE_PGXS dance in
src/test/modules?
ENOCLUE
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
On 2022-Feb-23, Andres Freund wrote:
Separately: I don't really understand why we do the whole if USE_PGXS dance in
src/test/modules?
Yeah, it seems a bit silly. I'm not opposed to making these makefiles
unconditionally do the PGXS thing -- or the non-PGXS thing, if that
makes it easier to build multiple binaries.
Agreed, we could easily lose that. I think we only do it in contrib
to serve as an example of how to use PGXS ... although considering
we're not *testing* that build mode, one wonders how much that proves.
In any case, src/test/modules doesn't have to do it.
regards, tom lane
On 23.02.22 21:30, Andres Freund wrote:
hen verifying that the meson port actually runs all perl based tests I came
across src/interfaces/libpq/test/regress.pl. Instead of running tests yet
another way, it seems better to convert it to a tap test.I hope others agree?
Where would we want that test to live? Right now we have the slightly odd
convention that some tap tests live in src/test/{misc,modules,...}. But
e.g. frontend binary ones are below src/bin/.
libpq TAP tests should be in src/interfaces/libpq/t/.
I think there were issues that the build farm wouldn't pick up a test
located there, but that should be fixed rather than worked around.
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 23.02.22 21:30, Andres Freund wrote:
Where would we want that test to live? Right now we have the slightly odd
convention that some tap tests live in src/test/{misc,modules,...}. But
e.g. frontend binary ones are below src/bin/.
libpq TAP tests should be in src/interfaces/libpq/t/.
I think there were issues that the build farm wouldn't pick up a test
located there, but that should be fixed rather than worked around.
That's failing to account for the fact that a libpq test can't
really be a pure-perl TAP test; you need some C code to drive the
library. I don't agree with intermixing such code with libpq
itself, independently of any buildsystem issues (which there
might well be). So I think the design of putting such tests under
src/modules is fine.
regards, tom lane
On 23.02.22 23:58, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 23.02.22 21:30, Andres Freund wrote:
Where would we want that test to live? Right now we have the slightly odd
convention that some tap tests live in src/test/{misc,modules,...}. But
e.g. frontend binary ones are below src/bin/.libpq TAP tests should be in src/interfaces/libpq/t/.
I think there were issues that the build farm wouldn't pick up a test
located there, but that should be fixed rather than worked around.That's failing to account for the fact that a libpq test can't
really be a pure-perl TAP test; you need some C code to drive the
library. I don't agree with intermixing such code with libpq
itself, independently of any buildsystem issues (which there
might well be).
Such things could be put under src/interfaces/libpq/test, or some other
subdirectory. We already have src/interfaces/ecpg/test.
So I think the design of putting such tests under
src/modules is fine.
I don't get what the rationale for that would be. libpq tests are not
"modules" of any kind.
If I'm working on libpq, I want to do make && make check inside the
libpq source directory.
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 23.02.22 23:58, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
libpq TAP tests should be in src/interfaces/libpq/t/.
That's failing to account for the fact that a libpq test can't
really be a pure-perl TAP test; you need some C code to drive the
library.
Such things could be put under src/interfaces/libpq/test, or some other
subdirectory. We already have src/interfaces/ecpg/test.
OK, but then the TAP scripts are under src/interfaces/libpq/test/t,
which isn't what you said. I have no great objection to moving
src/test/modules/libpq_pipeline/ to src/interfaces/libpq/test/,
though, as long as the buildfarm will cope.
regards, tom lane
On 2/23/22 20:52, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 23.02.22 23:58, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
libpq TAP tests should be in src/interfaces/libpq/t/.
That's failing to account for the fact that a libpq test can't
really be a pure-perl TAP test; you need some C code to drive the
library.Such things could be put under src/interfaces/libpq/test, or some other
subdirectory. We already have src/interfaces/ecpg/test.OK, but then the TAP scripts are under src/interfaces/libpq/test/t,
which isn't what you said. I have no great objection to moving
src/test/modules/libpq_pipeline/ to src/interfaces/libpq/test/,
though, as long as the buildfarm will cope.
It won't without some adjustment.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On 24.02.22 02:52, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
On 23.02.22 23:58, Tom Lane wrote:
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
libpq TAP tests should be in src/interfaces/libpq/t/.
That's failing to account for the fact that a libpq test can't
really be a pure-perl TAP test; you need some C code to drive the
library.Such things could be put under src/interfaces/libpq/test, or some other
subdirectory. We already have src/interfaces/ecpg/test.OK, but then the TAP scripts are under src/interfaces/libpq/test/t,
which isn't what you said. I have no great objection to moving
src/test/modules/libpq_pipeline/ to src/interfaces/libpq/test/,
though, as long as the buildfarm will cope.
I think the TAP scripts should be in src/interfaces/libpq/t/, as usual.
The supporting code snippets could live in some other directory under
src/interfaces/libpq/, which might be called "test" or something else,
not that important.
I think we should pick a layout that is proper and future-proof and then
adjust the buildfarm client as necessary. The issue of writing
libpq-specific tests has come up a few times recently; I think it would
be worth finding a proper solution to this that would facilitate that
work in the future.
Hi,
On 2022-02-24 13:31:40 +0100, Peter Eisentraut wrote:
I think the TAP scripts should be in src/interfaces/libpq/t/, as usual. The
supporting code snippets could live in some other directory under
src/interfaces/libpq/, which might be called "test" or something else, not
that important.
Why not in t/? We can't easily build the test programs in libpq/ itself, but
libpq/t should be fairly doable.
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
On 2022-02-24 13:31:40 +0100, Peter Eisentraut wrote:
I think the TAP scripts should be in src/interfaces/libpq/t/, as usual. The
supporting code snippets could live in some other directory under
src/interfaces/libpq/, which might be called "test" or something else, not
that important.
Why not in t/? We can't easily build the test programs in libpq/ itself, but
libpq/t should be fairly doable.
I think that having t/ directories contain only Perl test scripts
is a good convention that we should stick to. Peter's proposal
of a separate test/ subdirectory for C test scaffolding is
probably fine.
regards, tom lane
Hi,
On 2022-02-24 10:17:28 -0500, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
On 2022-02-24 13:31:40 +0100, Peter Eisentraut wrote:
I think the TAP scripts should be in src/interfaces/libpq/t/, as usual. The
supporting code snippets could live in some other directory under
src/interfaces/libpq/, which might be called "test" or something else, not
that important.Why not in t/? We can't easily build the test programs in libpq/ itself, but
libpq/t should be fairly doable.I think that having t/ directories contain only Perl test scripts
is a good convention that we should stick to. Peter's proposal
of a separate test/ subdirectory for C test scaffolding is
probably fine.
That exists today and continues to exist in the patch upthread, so it's easy
;). I just need to move the libpq/test/t to libpq/t and adjust the binary
path.
One annoying bit is that our current tap invocation infrastructure for msvc
won't know how to deal with that. We put the build directory containing t/
onto PATH, but that won't work for test/. But we also don't want to install
test binaries. Not sure what the solution for that is.
Even on !windows, we only know how to find "test executables" in tap tests via
PATH. We're in the source dir, so we can't just do test/executable.
I probably just need another coffee, but right now I don't even see how to add
anything to PATH given $(prove_check)'s definition - it ends up with multiple
shells. We can fix that by using && in the definition, which might be a good
thing anyway?
Attached three patches:
0001: Convert src/interfaces/libpq/test to a tap test
0002: Add tap test infrastructure to src/interfaces/libpq
0003: Move libpq_pipeline test into src/interfaces/libpq.
I did 0001 and 0002 in that order because prove errors out with a stacktrace
if no tap tests exist... It might make more sense to commit them together, but
for review it's easier to keep them separate I think.
Andrew, do you have an idea about the feasibility of supporting any of this
with the msvc build?
I'm mildly inclined to only do 0001 and 0002 for now. We'd not loose msvc
coverage, because it already doesn't build the test. Once we've ironed that
stuff out, we could do 0003?
Greetings,
Andres Freund
Attachments:
v2-0001-Convert-src-interfaces-libpq-test-to-a-tap-test.patchtext/x-diff; charset=us-asciiDownload+267-308
v2-0002-Add-tap-test-infrastructure-to-src-interfaces-lib.patchtext/x-diff; charset=us-asciiDownload+17-10
v2-0003-Move-libpq_pipeline-test-into-src-interfaces-libp.patchtext/x-diff; charset=us-asciiDownload+3-34
On Thu, 2022-02-24 at 08:46 -0800, Andres Freund wrote:
One annoying bit is that our current tap invocation infrastructure for msvc
won't know how to deal with that. We put the build directory containing t/
onto PATH, but that won't work for test/. But we also don't want to install
test binaries. Not sure what the solution for that is.
Would it help if the C executable, not Perl, was the thing actually
producing the TAP output? The binaries built from test/ could be placed
into t/. Or does that just open up a new set of problems?
--Jacob
Hi,
On 2022-02-24 17:03:33 +0000, Jacob Champion wrote:
On Thu, 2022-02-24 at 08:46 -0800, Andres Freund wrote:
One annoying bit is that our current tap invocation infrastructure for msvc
won't know how to deal with that. We put the build directory containing t/
onto PATH, but that won't work for test/. But we also don't want to install
test binaries. Not sure what the solution for that is.Would it help if the C executable, not Perl, was the thing actually
producing the TAP output? The binaries built from test/ could be placed
into t/. Or does that just open up a new set of problems?
I don't think it would help that much. And for the libpq pipeline test it
doesn't really make sense anyway, because we intentionally use it with
different traces and such.
I've thought about a few C tap tests that I'd like, but I think that's a
separate discussion.
Greetings,
Andres Freund
