Can we include capturing logs of pgdata/pg_upgrade_output.d/*/log in buildfarm
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
Hi,
Recently there have been few upgrade tap test failures in buildfarm
like in [1]https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2024-02-10%2007%3A03%3A10 & [2]https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2023-12-07%2003%3A56%3A20. Analysing these failures requires the log files
that are getting generated from src/bin/pg_upgrade at the following
locations:
tmp_check/*/pgdata/pg_upgrade_output.d/*/*.txt - e.g.
tmp_check/t_004_subscription_new_sub1_data/pgdata/pg_upgrade_output.d/20240214T052229.045/subs_invalid.txt
tmp_check/*/pgdata/pg_upgrade_output.d/*/*/*.log - e.g.
tmp_check/t_004_subscription_new_sub1_data/pgdata/pg_upgrade_output.d/20240214T052229.045/log/pg_upgrade_server.log
First regex is the testname_clusterinstance_data, second regex is the
timestamp used for pg_upgrade, third regex is for the text files
generated by pg_upgrade and fourth regex is for the log files
generated by pg_upgrade.
Can we include these log files also in the buildfarm?
[1]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2024-02-10%2007%3A03%3A10
[2]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2023-12-07%2003%3A56%3A20
Regards,
Vignesh
On Wed, Feb 14, 2024 at 03:51:08PM +0530, vignesh C wrote:
First regex is the testname_clusterinstance_data, second regex is the
timestamp used for pg_upgrade, third regex is for the text files
generated by pg_upgrade and fourth regex is for the log files
generated by pg_upgrade.Can we include these log files also in the buildfarm?
[1] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2024-02-10%2007%3A03%3A10
[2] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2023-12-07%2003%3A56%3A20
Indeed, these lack some patterns. Why not sending a pull request
around [1]https://github.com/PGBuildFarm/client-code/blob/main/PGBuild/Modules/TestUpgrade.pm -- Michael to get more patterns covered?
[1]: https://github.com/PGBuildFarm/client-code/blob/main/PGBuild/Modules/TestUpgrade.pm -- Michael
--
Michael
On Thu, 15 Feb 2024 at 07:24, Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Feb 14, 2024 at 03:51:08PM +0530, vignesh C wrote:
First regex is the testname_clusterinstance_data, second regex is the
timestamp used for pg_upgrade, third regex is for the text files
generated by pg_upgrade and fourth regex is for the log files
generated by pg_upgrade.Can we include these log files also in the buildfarm?
[1] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2024-02-10%2007%3A03%3A10
[2] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2023-12-07%2003%3A56%3A20Indeed, these lack some patterns. Why not sending a pull request
around [1] to get more patterns covered?
I have added a few more patterns to include the pg_upgrade generated
files. The attached patch has the changes for the same.
Adding Andrew also to get his thoughts on this.
Regards,
Vignesh
Attachments:
Include_pg_upgrade_generated_logs.patchtext/x-patch; charset=US-ASCII; name=Include_pg_upgrade_generated_logs.patchDownload+2-0
On Thu, 15 Feb 2024 at 08:36, vignesh C <vignesh21@gmail.com> wrote:
On Thu, 15 Feb 2024 at 07:24, Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Feb 14, 2024 at 03:51:08PM +0530, vignesh C wrote:
First regex is the testname_clusterinstance_data, second regex is the
timestamp used for pg_upgrade, third regex is for the text files
generated by pg_upgrade and fourth regex is for the log files
generated by pg_upgrade.Can we include these log files also in the buildfarm?
[1] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2024-02-10%2007%3A03%3A10
[2] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2023-12-07%2003%3A56%3A20Indeed, these lack some patterns. Why not sending a pull request
around [1] to get more patterns covered?I have added a few more patterns to include the pg_upgrade generated
files. The attached patch has the changes for the same.
Adding Andrew also to get his thoughts on this.
I have added the following commitfest entry for this:
https://commitfest.postgresql.org/47/4850/
Regards,
Vignesh
On 2024-02-25 Su 11:18, vignesh C wrote:
On Thu, 15 Feb 2024 at 08:36, vignesh C <vignesh21@gmail.com> wrote:
On Thu, 15 Feb 2024 at 07:24, Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Feb 14, 2024 at 03:51:08PM +0530, vignesh C wrote:
First regex is the testname_clusterinstance_data, second regex is the
timestamp used for pg_upgrade, third regex is for the text files
generated by pg_upgrade and fourth regex is for the log files
generated by pg_upgrade.Can we include these log files also in the buildfarm?
[1] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2024-02-10%2007%3A03%3A10
[2] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2023-12-07%2003%3A56%3A20Indeed, these lack some patterns. Why not sending a pull request
around [1] to get more patterns covered?I have added a few more patterns to include the pg_upgrade generated
files. The attached patch has the changes for the same.
Adding Andrew also to get his thoughts on this.I have added the following commitfest entry for this:
https://commitfest.postgresql.org/47/4850/
Buildfarm code patches do not belong in the Commitfest, I have marked
the item as rejected. You can send me patches directly or add a PR to
the buildfarm's github repo.
In this case the issue on drongo was a typo, the fix for which I had
forgotten to propagate back in December. Note that the buildfarm's
TestUpgrade.pm module is only used for branches < 15. For branches >= 15
we run the standard TAP test and this module does nothing.
More generally, the collection of logs etc. for pg_upgrade will improve
with the next release, which will be soon after I return from a vacation
in about 2 weeks - experience shows that making releases just before a
vacation is not a good idea :-)
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On Mon, 26 Feb 2024 at 10:57, Andrew Dunstan <andrew@dunslane.net> wrote:
On 2024-02-25 Su 11:18, vignesh C wrote:
On Thu, 15 Feb 2024 at 08:36, vignesh C <vignesh21@gmail.com> wrote:
On Thu, 15 Feb 2024 at 07:24, Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Feb 14, 2024 at 03:51:08PM +0530, vignesh C wrote:
First regex is the testname_clusterinstance_data, second regex is the
timestamp used for pg_upgrade, third regex is for the text files
generated by pg_upgrade and fourth regex is for the log files
generated by pg_upgrade.Can we include these log files also in the buildfarm?
[1] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2024-02-10%2007%3A03%3A10
[2] - https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=drongo&dt=2023-12-07%2003%3A56%3A20Indeed, these lack some patterns. Why not sending a pull request
around [1] to get more patterns covered?I have added a few more patterns to include the pg_upgrade generated
files. The attached patch has the changes for the same.
Adding Andrew also to get his thoughts on this.I have added the following commitfest entry for this:
https://commitfest.postgresql.org/47/4850/Buildfarm code patches do not belong in the Commitfest, I have marked
the item as rejected. You can send me patches directly or add a PR to
the buildfarm's github repo.
Ok, I will send over the patch directly for the required things.
In this case the issue on drongo was a typo, the fix for which I had
forgotten to propagate back in December. Note that the buildfarm's
TestUpgrade.pm module is only used for branches < 15. For branches >= 15
we run the standard TAP test and this module does nothing.More generally, the collection of logs etc. for pg_upgrade will improve
with the next release, which will be soon after I return from a vacation
in about 2 weeks - experience shows that making releases just before a
vacation is not a good idea :-)
Thanks, that will be helpful.
Regards,
Vignesh