automating perl compile time checking
Here is a follow up patch to last weeks commit allowing all perl files
to be checked clean for compile time errors and warnings.
The patch contains a simple script to run the checks. The code that
finds perl files is put in a function in a single file that is sourced
by the three locations that need it.
The directory pgperlcritic is renamed to perlcheck, as it not contains
the new script as well as pgperlcritic.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
0001-Extend-and-slightly-refactor-perl-checking.patchtext/x-patch; name=0001-Extend-and-slightly-refactor-perl-checking.patchDownload
From 6247cd1fbba16426bb1745521d7b444d60ebbd0a Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <andrew@dunslane.net>
Date: Tue, 5 Jun 2018 10:25:55 -0400
Subject: [PATCH] Extend and slightly refactor perl checking
A script is added to check for perl compile time errors and warnings.
The code to detect all the perl files is put in a central place and
included in the three places it is now needed. The directory
pgperlcritic is renamed to perlcheck, and contains the new check script
as well as pgperlcritic.
---
src/tools/perlcheck/find_perl_files | 15 ++++++++++++
src/tools/{pgperlcritic => perlcheck}/perlcriticrc | 0
src/tools/perlcheck/pgperlcritic | 20 ++++++++++++++++
src/tools/perlcheck/pgperlsyncheck | 17 +++++++++++++
src/tools/pgindent/pgperltidy | 14 +++--------
src/tools/pgperlcritic/pgperlcritic | 28 ----------------------
6 files changed, 55 insertions(+), 39 deletions(-)
create mode 100644 src/tools/perlcheck/find_perl_files
rename src/tools/{pgperlcritic => perlcheck}/perlcriticrc (100%)
create mode 100755 src/tools/perlcheck/pgperlcritic
create mode 100755 src/tools/perlcheck/pgperlsyncheck
delete mode 100755 src/tools/pgperlcritic/pgperlcritic
diff --git a/src/tools/perlcheck/find_perl_files b/src/tools/perlcheck/find_perl_files
new file mode 100644
index 0000000..e10466a
--- /dev/null
+++ b/src/tools/perlcheck/find_perl_files
@@ -0,0 +1,15 @@
+
+# src/tools/perlcheck/find_perl_files
+
+# shell function to find all perl files in the source tree
+
+find_perl_files () {
+ {
+ # take all .pl and .pm files
+ find . -type f -name '*.p[lm]' -print
+ # take executable files that file(1) thinks are perl files
+ find . -type f -perm -100 -exec file {} \; -print |
+ egrep -i ':.*perl[0-9]*\>' |
+ cut -d: -f1
+ } | sort -u
+}
diff --git a/src/tools/pgperlcritic/perlcriticrc b/src/tools/perlcheck/perlcriticrc
similarity index 100%
rename from src/tools/pgperlcritic/perlcriticrc
rename to src/tools/perlcheck/perlcriticrc
diff --git a/src/tools/perlcheck/pgperlcritic b/src/tools/perlcheck/pgperlcritic
new file mode 100755
index 0000000..6a31f67
--- /dev/null
+++ b/src/tools/perlcheck/pgperlcritic
@@ -0,0 +1,20 @@
+#!/bin/sh
+
+# src/tools/pgperlcritic/pgperlcritic
+
+test -f src/tools/perlcheck/perlcriticrc || {
+ echo could not find src/tools/perlcheck/perlcriticrc
+ exit 1
+ }
+
+set -e
+
+# set this to override default perlcritic program:
+PERLCRITIC=${PERLCRITIC:-perlcritic}
+
+. src/tools/perlcheck/find_perl_files
+
+find_perl_files | xargs $PERLCRITIC \
+ --quiet \
+ --program-extensions .pl \
+ --profile=src/tools/perlcheck/perlcriticrc
diff --git a/src/tools/perlcheck/pgperlsyncheck b/src/tools/perlcheck/pgperlsyncheck
new file mode 100755
index 0000000..4595d16
--- /dev/null
+++ b/src/tools/perlcheck/pgperlsyncheck
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+# script to detect compile time errors and warnings in all perl files
+
+INCLUDES="-I src/tools/msvc -I src/tools/msvc/dummylib -I src/backend/catalog"
+INCLUDES="-I src/test/perl -I src/backend/utils/mb/Unicode $INCLUDES"
+INCLUDES="-I src/bin/pg_rewind -I src/test/ssl $INCLUDES"
+
+set -e
+
+. src/tools/perlcheck/find_perl_files
+
+# for zsh
+setopt shwordsplit 2>/dev/null || true
+
+find_perl_files | xargs -L 1 perl $INCLUDES -cw 2>&1 | grep -v OK
+
diff --git a/src/tools/pgindent/pgperltidy b/src/tools/pgindent/pgperltidy
index 5d9aa7c..5e70411 100755
--- a/src/tools/pgindent/pgperltidy
+++ b/src/tools/pgindent/pgperltidy
@@ -7,14 +7,6 @@ set -e
# set this to override default perltidy program:
PERLTIDY=${PERLTIDY:-perltidy}
-# locate all Perl files in the tree
-(
- # take all .pl and .pm files
- find . -type f -a \( -name '*.pl' -o -name '*.pm' \)
- # take executable files that file(1) thinks are perl files
- find . -type f -perm -100 -exec file {} \; |
- egrep -i ':.*perl[0-9]*\>' |
- cut -d: -f1
-) |
-sort -u |
-xargs $PERLTIDY --profile=src/tools/pgindent/perltidyrc
+. src/tools/perlcheck/find_perl_files
+
+find_perl_files | xargs $PERLTIDY --profile=src/tools/pgindent/perltidyrc
diff --git a/src/tools/pgperlcritic/pgperlcritic b/src/tools/pgperlcritic/pgperlcritic
deleted file mode 100755
index 28264b1..0000000
--- a/src/tools/pgperlcritic/pgperlcritic
+++ /dev/null
@@ -1,28 +0,0 @@
-#!/bin/sh
-
-# src/tools/pgperlcritic/pgperlcritic
-
-test -f src/tools/pgperlcritic/perlcriticrc || {
- echo could not find src/tools/pgperlcritic/perlcriticrc
- exit 1
- }
-
-set -e
-
-# set this to override default perlcritic program:
-PERLCRITIC=${PERLCRITIC:-perlcritic}
-
-# locate all Perl files in the tree
-{
- # take all .pl and .pm files
- find . -type f -name '*.p[lm]' -print
- # take executable files that file(1) thinks are perl files
- find . -type f -perm -100 -exec file {} \; -print |
- egrep -i ':.*perl[0-9]*\>' |
- cut -d: -f1
-} |
-sort -u |
-xargs $PERLCRITIC \
- --quiet \
- --program-extensions .pl \
- --profile=src/tools/pgperlcritic/perlcriticrc
--
2.9.5
On 5 Jun 2018, at 16:31, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote:
The patch contains a simple script to run the checks. The code that finds perl files is put in a function in a single file that is sourced by the three locations that need it.
+1 on centralizing the find-files function.
The directory pgperlcritic is renamed to perlcheck, as it not contains the new script as well as pgperlcritic.
This comment should say “perlcheck/..” and not “pgperlcritic/.." I assume:
--- /dev/null
+++ b/src/tools/perlcheck/pgperlcritic
@@ -0,0 +1,20 @@
+#!/bin/sh
+
+# src/tools/pgperlcritic/pgperlcriticcheers ./daniel
On 06/05/2018 05:14 PM, Daniel Gustafsson wrote:
This comment should say “perlcheck/..” and not “pgperlcritic/.." I assume:
Thanks. will fix.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 2018-Jun-05, Daniel Gustafsson wrote:
On 5 Jun 2018, at 16:31, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote:
The patch contains a simple script to run the checks. The code that finds perl files is put in a function in a single file that is sourced by the three locations that need it.
+1 on centralizing the find-files function.
+1 on that. Why do we need to make the new find_perl_files file
executable, given it's always sourced? (I would have given a .sh
extension because it's a lib not an executable, but I suppose that's
just matter of taste; we certainly don't have a policy about it).
Looks fine to me either way.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 06/11/2018 02:33 PM, Alvaro Herrera wrote:
On 2018-Jun-05, Daniel Gustafsson wrote:
On 5 Jun 2018, at 16:31, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote:
The patch contains a simple script to run the checks. The code that finds perl files is put in a function in a single file that is sourced by the three locations that need it.+1 on centralizing the find-files function.
+1 on that. Why do we need to make the new find_perl_files file
executable, given it's always sourced? (I would have given a .sh
extension because it's a lib not an executable, but I suppose that's
just matter of taste; we certainly don't have a policy about it).Looks fine to me either way.
I've committed this, but I'm fine if people want to tweak the names. It
probably doesn't need to be executable.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 6/11/18 15:38, Andrew Dunstan wrote:
On 5 Jun 2018, at 16:31, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote:
The patch contains a simple script to run the checks. The code that finds perl files is put in a function in a single file that is sourced by the three locations that need it.+1 on centralizing the find-files function.
+1 on that. Why do we need to make the new find_perl_files file
executable, given it's always sourced? (I would have given a .sh
extension because it's a lib not an executable, but I suppose that's
just matter of taste; we certainly don't have a policy about it).Looks fine to me either way.
I've committed this, but I'm fine if people want to tweak the names. It
probably doesn't need to be executable.
Why is this being committed after feature freeze?
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 06/11/2018 04:01 PM, Peter Eisentraut wrote:
On 6/11/18 15:38, Andrew Dunstan wrote:
On 5 Jun 2018, at 16:31, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote:
The patch contains a simple script to run the checks. The code that finds perl files is put in a function in a single file that is sourced by the three locations that need it.+1 on centralizing the find-files function.
+1 on that. Why do we need to make the new find_perl_files file
executable, given it's always sourced? (I would have given a .sh
extension because it's a lib not an executable, but I suppose that's
just matter of taste; we certainly don't have a policy about it).Looks fine to me either way.
I've committed this, but I'm fine if people want to tweak the names. It
probably doesn't need to be executable.Why is this being committed after feature freeze?
This affects pretty much nothing. In fact some of the other changes I've
recently committed were arguably more dangerous. Do you want me to
revert the whole lot?
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
On 06/11/2018 04:01 PM, Peter Eisentraut wrote:
Why is this being committed after feature freeze?
This affects pretty much nothing. In fact some of the other changes I've
recently committed were arguably more dangerous. Do you want me to
revert the whole lot?
AFAICS this is adding testing, not "features", so it seems fine from here.
regards, tom lane
On 06/11/2018 03:38 PM, Andrew Dunstan wrote:
On 06/11/2018 02:33 PM, Alvaro Herrera wrote:
On 2018-Jun-05, Daniel Gustafsson wrote:
On 5 Jun 2018, at 16:31, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
The patch contains a simple script to run the checks. The code that
finds perl files is put in a function in a single file that is
sourced by the three locations that need it.+1 on centralizing the find-files function.
+1 on that. Why do we need to make the new find_perl_files file
executable, given it's always sourced? (I would have given a .sh
extension because it's a lib not an executable, but I suppose that's
just matter of taste; we certainly don't have a policy about it).Looks fine to me either way.
I've committed this, but I'm fine if people want to tweak the names.
It probably doesn't need to be executable.
People might be interested to see the perlcritic and 'perl -cw" checks
in operation:
The module isn't actually using the scripts in src/tools/perlcheck,
because they are designed to be quiet and it's designed to be more
verbose, but apart from that it's doing exactly the same thing.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 6/11/18 16:06, Andrew Dunstan wrote:
This affects pretty much nothing. In fact some of the other changes I've
recently committed were arguably more dangerous. Do you want me to
revert the whole lot?
No, but this whole let's clean up the Perl code initiative seemed to
have come out of nowhere after feature freeze. The Perl code was fine
before, so if we're going to be polishing it around it should go into
the next release. I would actually have liked to have had more time to
discuss some of the changes, since I didn't seem to agree to some of
them at first reading.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 06/15/2018 12:21 AM, Peter Eisentraut wrote:
On 6/11/18 16:06, Andrew Dunstan wrote:
This affects pretty much nothing. In fact some of the other changes I've
recently committed were arguably more dangerous. Do you want me to
revert the whole lot?No, but this whole let's clean up the Perl code initiative seemed to
have come out of nowhere after feature freeze. The Perl code was fine
before, so if we're going to be polishing it around it should go into
the next release. I would actually have liked to have had more time to
discuss some of the changes, since I didn't seem to agree to some of
them at first reading.
There were changes made for perlcritic, but almost none for this check.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services