pgsql: Enable TAP tests of pg_receivewal for ZLIB on Windows, take thre

Started by Michael Paquieralmost 5 years ago7 messagescomitters
Jump to latest
#1Michael Paquier
michael@paquier.xyz

Enable TAP tests of pg_receivewal for ZLIB on Windows, take three

This reverts commit 6a2c532. fairywren and bowerbird failed those tests
because of incorrect versions of ZLIB linked to, causing errors like
SIGBREAKs that stopped buildfarm runs or EACCES failures when writing
compressed WAL segments.

Andrew Dunstan has done all the investigation here, so he deserves all
the credit for being able to enable those tests on Windows.

Discussion: /messages/by-id/9040d5ed-6462-66a4-07ac-2923785ae563@dunslane.net

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/0b8ea707580640f9d5562de77f4b563fa9131547

Modified Files
--------------
src/bin/pg_basebackup/t/020_pg_receivewal.pl | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)

#2Andrew Dunstan
andrew@dunslane.net
In reply to: Michael Paquier (#1)
Re: pgsql: Enable TAP tests of pg_receivewal for ZLIB on Windows, take thre

On 7/30/21 9:28 PM, Michael Paquier wrote:

Enable TAP tests of pg_receivewal for ZLIB on Windows, take three

This reverts commit 6a2c532. fairywren and bowerbird failed those tests
because of incorrect versions of ZLIB linked to, causing errors like
SIGBREAKs that stopped buildfarm runs or EACCES failures when writing
compressed WAL segments.

Andrew Dunstan has done all the investigation here, so he deserves all
the credit for being able to enable those tests on Windows.

And now jacana is unhappy. I'll get it fixed shortly. If it ends up that
we have somehow found a situation where Msys/Mingw and MSVC can no
longer share a zlib implementation I'm going to be unhappy too.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

#3Michael Paquier
michael@paquier.xyz
In reply to: Andrew Dunstan (#2)
Re: pgsql: Enable TAP tests of pg_receivewal for ZLIB on Windows, take thre

On Sat, Jul 31, 2021 at 07:16:22AM -0400, Andrew Dunstan wrote:

And now jacana is unhappy. I'll get it fixed shortly. If it ends up that
we have somehow found a situation where Msys/Mingw and MSVC can no
longer share a zlib implementation I'm going to be unhappy too.

In the worse case, could it be possible to do things so as we skip
tests only with $is_msys2 or a new flag? It would be really nice to
keep some coverage here for some of the Windows builds.
--
Michael

#4Andrew Dunstan
andrew@dunslane.net
In reply to: Michael Paquier (#3)
Re: pgsql: Enable TAP tests of pg_receivewal for ZLIB on Windows, take thre

On 7/31/21 11:32 PM, Michael Paquier wrote:

On Sat, Jul 31, 2021 at 07:16:22AM -0400, Andrew Dunstan wrote:

And now jacana is unhappy. I'll get it fixed shortly. If it ends up that
we have somehow found a situation where Msys/Mingw and MSVC can no
longer share a zlib implementation I'm going to be unhappy too.

In the worse case, could it be possible to do things so as we skip
tests only with $is_msys2 or a new flag? It would be really nice to
keep some coverage here for some of the Windows builds.

It won't be necessary.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

#5Michael Paquier
michael@paquier.xyz
In reply to: Andrew Dunstan (#4)
Re: pgsql: Enable TAP tests of pg_receivewal for ZLIB on Windows, take thre

On Sun, Aug 01, 2021 at 08:09:47AM -0400, Andrew Dunstan wrote:

It won't be necessary.

These buildfarm members are all green, thanks! Did you split the
version of ZLIB used for each build?
--
Michael

#6Andrew Dunstan
andrew@dunslane.net
In reply to: Michael Paquier (#5)
Re: pgsql: Enable TAP tests of pg_receivewal for ZLIB on Windows, take thre

On 8/5/21 2:02 AM, Michael Paquier wrote:

On Sun, Aug 01, 2021 at 08:09:47AM -0400, Andrew Dunstan wrote:

It won't be necessary.

These buildfarm members are all green, thanks! Did you split the
version of ZLIB used for each build?

Yeah, pretty much. I wonder what zlib API the pg_receivewal tests are
tickling that other tests have not?

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

#7Michael Paquier
michael@paquier.xyz
In reply to: Andrew Dunstan (#6)
Re: pgsql: Enable TAP tests of pg_receivewal for ZLIB on Windows, take thre

On Thu, Aug 05, 2021 at 11:49:01AM -0400, Andrew Dunstan wrote:

Yeah, pretty much. I wonder what zlib API the pg_receivewal tests are
tickling that other tests have not?

The main issue is that this is the first test suite stressing directly
zlib internals, as we only use gzread(), gzwrite() and gzflush()
within pg_dump/ and pg_basebackup/. And there is currently no
coverage of pg_dump --compress. My guess is that nobody tackled this
issue because of the need to make the tests conditional depending on
the compilation options.

Saying that, I recall two error patterns from this thread's
experience thanks to the specific error messages we have in
pg_receivewal for the handling of the WAL segments:
- gzopen(), followed by gzwrite() failing on EACCES.
- gzopen(), followed by gzflush() failing.
--
Michael