pg_verify_checksums review

Started by Daniel Gustafssonover 7 years ago3 messages
#1Daniel Gustafsson
daniel@yesql.se
3 attachment(s)

In looking over pg_verify_checksums I found a few small things that I think
would improve on it:

* pg_verify_checksums was placed in the Client Utils section in the docs.
Since it requries physical access to the cluster datafiles it seems to belong
in the Server Utils section.

* The -D option and supported environment variable wasn’t documented.

* Only -D is supported for specifying the data directory, but most all other
utilities also support --pgdata on top of -D. To present a consistent user
interface we should probably support --pgdata in pg_verify_checksums as well.

The latter is I assume too invasive as we are past the freeze, but the first
two docs patches would make sense in 11 IMO as they document whats in the tree.

The attached patches fixes the above mentioned things (I don’t have a docs
toolchain working right now so the docs patches are best effort).

cheers ./daniel

Attachments:

0001-Move-pg_verify_checksum-from-client-to-server-utils.patchapplication/octet-stream; name=0001-Move-pg_verify_checksum-from-client-to-server-utils.patch; x-unix-mode=0644Download
From ca6efd2f09e075022a15d0f15f131d3711a7c728 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Tue, 19 Jun 2018 00:21:30 +0200
Subject: [PATCH 1/3] Move pg_verify_checksum from client to server utils

The pg_verify_checksum utility requires physical access to the
datafiles of an offline cluster, so it rather belongs in the
section for Server utils as opposed to the current placement in
Client utils.
---
 doc/src/sgml/reference.sgml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index 73ef212c08..db4f4167e3 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -255,7 +255,6 @@
    &pgReceivewal;
    &pgRecvlogical;
    &pgRestore;
-   &pgVerifyChecksums;
    &psqlRef;
    &reindexdb;
    &vacuumdb;
@@ -284,6 +283,7 @@
    &pgtestfsync;
    &pgtesttiming;
    &pgupgrade;
+   &pgVerifyChecksums;
    &pgwaldump;
    &postgres;
    &postmaster;
-- 
2.14.1.145.gb3622a4ee

0002-Add-D-option-to-pg_verify_checksum-documentation.patchapplication/octet-stream; name=0002-Add-D-option-to-pg_verify_checksum-documentation.patch; x-unix-mode=0644Download
From dd5430df91600e6e464298ac7b3fa76d83debd54 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Tue, 19 Jun 2018 00:46:51 +0200
Subject: [PATCH 2/3] Add -D option to pg_verify_checksum documentation

While listed in the Synopsis and technically not required, it's more
consistent to also list it in the Options section.
---
 doc/src/sgml/ref/pg_verify_checksums.sgml | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/doc/src/sgml/ref/pg_verify_checksums.sgml b/doc/src/sgml/ref/pg_verify_checksums.sgml
index 1a40609951..3416955ec9 100644
--- a/doc/src/sgml/ref/pg_verify_checksums.sgml
+++ b/doc/src/sgml/ref/pg_verify_checksums.sgml
@@ -43,6 +43,15 @@ PostgreSQL documentation
 
     <variablelist>
 
+     <varlistentry>
+      <term><option>-D <replaceable>directory</replaceable></option></term>
+      <listitem>
+       <para>
+        Specifies the directory where the database cluster is stored.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry>
       <term><option>-r <replaceable>relfilenode</replaceable></option></term>
       <listitem>
@@ -85,6 +94,23 @@ PostgreSQL documentation
    </para>
  </refsect1>
 
+ <refsect1>
+  <title>Environment</title>
+
+  <variablelist>
+   <varlistentry>
+    <term><envar>PGDATA</envar></term>
+
+    <listitem>
+     <para>
+      Specifies the directory where the database cluster is 
+      stored; can be overridden using the <option>-D</option> option.
+     </para>
+    </listitem>
+   </varlistentry>
+  </variablelist>
+ </refsect1>
+
  <refsect1>
   <title>Notes</title>
   <para>
-- 
2.14.1.145.gb3622a4ee

0003-Add-pgdata-as-alias-for-D-for-consistency.patchapplication/octet-stream; name=0003-Add-pgdata-as-alias-for-D-for-consistency.patch; x-unix-mode=0644Download
From 9f39b8ee79acf35a6c910291759b5f6786b4e717 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Tue, 19 Jun 2018 00:53:39 +0200
Subject: [PATCH 3/3] Add --pgdata as alias for -D for consistency

Most, if not all, tools use both -D and --pgdata for specifying
the DataDir. Support both in pg_verify_checsums as well to be
consistent with the other tools.
---
 doc/src/sgml/ref/pg_verify_checksums.sgml         |  3 ++-
 src/bin/pg_verify_checksums/pg_verify_checksums.c | 18 ++++++++++++------
 2 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/doc/src/sgml/ref/pg_verify_checksums.sgml b/doc/src/sgml/ref/pg_verify_checksums.sgml
index 3416955ec9..c4c15a46ba 100644
--- a/doc/src/sgml/ref/pg_verify_checksums.sgml
+++ b/doc/src/sgml/ref/pg_verify_checksums.sgml
@@ -23,7 +23,7 @@ PostgreSQL documentation
   <cmdsynopsis>
    <command>pg_verify_checksums</command>
    <arg choice="opt"><replaceable class="parameter">option</replaceable></arg>
-   <arg choice="opt"><arg choice="opt"><option>-D</option></arg> <replaceable class="parameter">datadir</replaceable></arg>
+   <arg choice="opt"><arg choice="opt"><option>-D</option></arg><arg choice="opt"><option>--pgdata</option></arg> <replaceable class="parameter">datadir</replaceable></arg>
   </cmdsynopsis>
  </refsynopsisdiv>
 
@@ -45,6 +45,7 @@ PostgreSQL documentation
 
      <varlistentry>
       <term><option>-D <replaceable>directory</replaceable></option></term>
+      <term><option>--pgdata=<replaceable>directory</replaceable></option></term>
       <listitem>
        <para>
         Specifies the directory where the database cluster is stored.
diff --git a/src/bin/pg_verify_checksums/pg_verify_checksums.c b/src/bin/pg_verify_checksums/pg_verify_checksums.c
index 845d5aba27..48e754822c 100644
--- a/src/bin/pg_verify_checksums/pg_verify_checksums.c
+++ b/src/bin/pg_verify_checksums/pg_verify_checksums.c
@@ -41,11 +41,11 @@ usage()
 	printf(_("Usage:\n"));
 	printf(_("  %s [OPTION] [DATADIR]\n"), progname);
 	printf(_("\nOptions:\n"));
-	printf(_(" [-D] DATADIR    data directory\n"));
-	printf(_("  -r relfilenode check only relation with specified relfilenode\n"));
-	printf(_("  -d             debug output, listing all checked blocks\n"));
-	printf(_("  -V, --version  output version information, then exit\n"));
-	printf(_("  -?, --help     show this help, then exit\n"));
+	printf(_(" [-D, --pgdata=]DATADIR  data directory\n"));
+	printf(_("  -r relfilenode         check only relation with specified relfilenode\n"));
+	printf(_("  -d                     debug output, listing all checked blocks\n"));
+	printf(_("  -V, --version          output version information, then exit\n"));
+	printf(_("  -?, --help             show this help, then exit\n"));
 	printf(_("\nIf no data directory (DATADIR) is specified, "
 			 "the environment variable PGDATA\nis used.\n\n"));
 	printf(_("Report bugs to <pgsql-bugs@postgresql.org>.\n"));
@@ -205,8 +205,14 @@ scan_directory(char *basedir, char *subdir)
 int
 main(int argc, char *argv[])
 {
+	static struct option long_options[] = {
+		{"pgdata", required_argument, NULL, 'D'},
+		{NULL, 0, NULL, 0}
+	};
+
 	char	   *DataDir = NULL;
 	int			c;
+	int			option_index;
 	bool		crc_ok;
 
 	set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_verify_checksums"));
@@ -227,7 +233,7 @@ main(int argc, char *argv[])
 		}
 	}
 
-	while ((c = getopt(argc, argv, "D:r:d")) != -1)
+	while ((c = getopt_long(argc, argv, "D:r:d", long_options, &option_index)) != -1)
 	{
 		switch (c)
 		{
-- 
2.14.1.145.gb3622a4ee

#2Magnus Hagander
magnus@hagander.net
In reply to: Daniel Gustafsson (#1)
Re: pg_verify_checksums review

On Tue, Jun 19, 2018 at 10:25 AM, Daniel Gustafsson <daniel@yesql.se> wrote:

In looking over pg_verify_checksums I found a few small things that I think
would improve on it:

* pg_verify_checksums was placed in the Client Utils section in the docs.
Since it requries physical access to the cluster datafiles it seems to
belong
in the Server Utils section.

Makes sense.

* The -D option and supported environment variable wasn’t documented.

* Only -D is supported for specifying the data directory, but most all
other
utilities also support --pgdata on top of -D. To present a consistent user
interface we should probably support --pgdata in pg_verify_checksums as
well.

The latter is I assume too invasive as we are past the freeze, but the
first
two docs patches would make sense in 11 IMO as they document whats in the
tree.

The attached patches fixes the above mentioned things (I don’t have a docs
toolchain working right now so the docs patches are best effort).

I believe both those are fine for 11, so I've pushed that. I kept it as a
separate patch to make it easy enough to revert it if people prefer that :)

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/&gt;
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/&gt;

#3sixela
sixela@google.com
In reply to: Magnus Hagander (#2)
Re: pg_verify_checksums review

Hello,

There is a similar utility that I wrote that does offline checksum
verification as well.

https://github.com/google/pg_page_verification

This utility includes a verbose option as well as scanning multiple
subsequent segment files.

Alexis

--
Sent from: http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html