inconvenient compression options in pg_basebackup
There is no way to tell pg_basebackup to "just compress the thing in a
default way". You have to pick some number and then write -Z8 or
something. I suppose that interface was copied from pg_dump, but there
it's not that commonly used because the right formats are compressed by
default.
I suggest we add an argument-less option -z that means "compress", and
then -Z can be relegated to choosing the compression level.
On Thu, May 19, 2011 at 17:56, Peter Eisentraut <peter_e@gmx.net> wrote:
There is no way to tell pg_basebackup to "just compress the thing in a
default way". You have to pick some number and then write -Z8 or
something. I suppose that interface was copied from pg_dump, but there
it's not that commonly used because the right formats are compressed by
default.
It was copied from pg_dump, yes.
I suggest we add an argument-less option -z that means "compress", and
then -Z can be relegated to choosing the compression level.
We can't just use -Z without a parameter for that?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On fre, 2011-05-20 at 14:19 -0400, Magnus Hagander wrote:
I suggest we add an argument-less option -z that means "compress",
and
then -Z can be relegated to choosing the compression level.
We can't just use -Z without a parameter for that?
You can't portably have a command-line option with an optional argument.
On Fri, May 20, 2011 at 17:45, Peter Eisentraut <peter_e@gmx.net> wrote:
On fre, 2011-05-20 at 14:19 -0400, Magnus Hagander wrote:
I suggest we add an argument-less option -z that means "compress",
and
then -Z can be relegated to choosing the compression level.
We can't just use -Z without a parameter for that?
You can't portably have a command-line option with an optional argument.
Ugh.
In that case, I'm fine with your suggestion.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On sön, 2011-05-22 at 16:43 -0400, Magnus Hagander wrote:
On Fri, May 20, 2011 at 17:45, Peter Eisentraut <peter_e@gmx.net> wrote:
On fre, 2011-05-20 at 14:19 -0400, Magnus Hagander wrote:
I suggest we add an argument-less option -z that means "compress",
and
then -Z can be relegated to choosing the compression level.
We can't just use -Z without a parameter for that?
You can't portably have a command-line option with an optional argument.
Ugh.
In that case, I'm fine with your suggestion.
Quick patch for verification. I chose the naming -z/--gzip to mirror
GNU tar.
Attachments:
pg_basebackup-compress.patchtext/x-patch; charset=UTF-8; name=pg_basebackup-compress.patchDownload
diff --git i/doc/src/sgml/ref/pg_basebackup.sgml w/doc/src/sgml/ref/pg_basebackup.sgml
index 8a7b833..ce7eb52 100644
--- i/doc/src/sgml/ref/pg_basebackup.sgml
+++ w/doc/src/sgml/ref/pg_basebackup.sgml
@@ -169,8 +169,8 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
- <term><option>-Z <replaceable class="parameter">level</replaceable></option></term>
- <term><option>--compress=<replaceable class="parameter">level</replaceable></option></term>
+ <term><option>-z</option></term>
+ <term><option>--gzip</option></term>
<listitem>
<para>
Enables gzip compression of tar file output. Compression is only
@@ -179,6 +179,18 @@ PostgreSQL documentation
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><option>-Z <replaceable class="parameter">level</replaceable></option></term>
+ <term><option>--compress-level=<replaceable class="parameter">level</replaceable></option></term>
+ <listitem>
+ <para>
+ Sets the compression level when gzip compression is enabled.
+ The default is the default compression level of the zlib
+ library.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
<para>
@@ -394,11 +406,11 @@ PostgreSQL documentation
</para>
<para>
- To create a backup of the local server with one maximum compressed
+ To create a backup of the local server with one compressed
tar file for each tablespace, and store it in the directory
<filename>backup</filename>, showing a progress report while running:
<screen>
-<prompt>$</prompt> <userinput>pg_basebackup -D backup -Ft -Z9 -P</userinput>
+<prompt>$</prompt> <userinput>pg_basebackup -D backup -Ft -z -P</userinput>
</screen>
</para>
diff --git i/src/bin/pg_basebackup/pg_basebackup.c w/src/bin/pg_basebackup/pg_basebackup.c
index 1f31fe0..7c2cb57 100644
--- i/src/bin/pg_basebackup/pg_basebackup.c
+++ w/src/bin/pg_basebackup/pg_basebackup.c
@@ -32,7 +32,10 @@ char format = 'p'; /* p(lain)/t(ar) */
char *label = "pg_basebackup base backup";
bool showprogress = false;
int verbose = 0;
-int compresslevel = 0;
+bool gzip = false;
+#ifdef HAVE_LIBZ
+int compresslevel = Z_DEFAULT_COMPRESSION;
+#endif
bool includewal = false;
bool fastcheckpoint = false;
char *dbhost = NULL;
@@ -126,7 +129,8 @@ usage(void)
printf(_(" -D, --pgdata=DIRECTORY receive base backup into directory\n"));
printf(_(" -F, --format=p|t output format (plain, tar)\n"));
printf(_(" -x, --xlog include required WAL files in backup\n"));
- printf(_(" -Z, --compress=0-9 compress tar output\n"));
+ printf(_(" -z, --gzip compress tar output with gzip\n"));
+ printf(_(" -Z, --compress-level=0-9 compression level\n"));
printf(_("\nGeneral options:\n"));
printf(_(" -c, --checkpoint=fast|spread\n"
" set fast or spread checkpointing\n"));
@@ -265,7 +269,7 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
else
{
#ifdef HAVE_LIBZ
- if (compresslevel > 0)
+ if (gzip)
{
snprintf(fn, sizeof(fn), "%s/base.tar.gz", basedir);
ztarfile = gzopen(fn, "wb");
@@ -289,7 +293,7 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
* Specific tablespace
*/
#ifdef HAVE_LIBZ
- if (compresslevel > 0)
+ if (gzip)
{
snprintf(fn, sizeof(fn), "%s/%s.tar.gz", basedir, PQgetvalue(res, rownum, 0));
ztarfile = gzopen(fn, "wb");
@@ -309,7 +313,7 @@ ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
}
#ifdef HAVE_LIBZ
- if (compresslevel > 0)
+ if (gzip)
{
if (!ztarfile)
{
@@ -919,7 +923,8 @@ main(int argc, char **argv)
{"format", required_argument, NULL, 'F'},
{"checkpoint", required_argument, NULL, 'c'},
{"xlog", no_argument, NULL, 'x'},
- {"compress", required_argument, NULL, 'Z'},
+ {"gzip", no_argument, NULL, 'z'},
+ {"compress-level", required_argument, NULL, 'Z'},
{"label", required_argument, NULL, 'l'},
{"host", required_argument, NULL, 'h'},
{"port", required_argument, NULL, 'p'},
@@ -952,7 +957,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:F:l:Z:c:h:p:U:xwWvP",
+ while ((c = getopt_long(argc, argv, "D:F:l:c:h:p:U:xwWvPzZ:",
long_options, &option_index)) != -1)
{
switch (c)
@@ -978,6 +983,9 @@ main(int argc, char **argv)
case 'l':
label = xstrdup(optarg);
break;
+ case 'z':
+ gzip = true;
+ break;
case 'Z':
compresslevel = atoi(optarg);
if (compresslevel <= 0 || compresslevel > 9)
@@ -1058,7 +1066,7 @@ main(int argc, char **argv)
/*
* Mutually exclusive arguments
*/
- if (format == 'p' && compresslevel > 0)
+ if (format == 'p' && gzip)
{
fprintf(stderr,
_("%s: only tar mode backups can be compressed\n"),
@@ -1069,7 +1077,7 @@ main(int argc, char **argv)
}
#ifndef HAVE_LIBZ
- if (compresslevel > 0)
+ if (gzip)
{
fprintf(stderr,
_("%s: this build does not support compression\n"),
@@ -1077,7 +1085,7 @@ main(int argc, char **argv)
exit(1);
}
#else
- if (compresslevel > 0 && strcmp(basedir, "-") == 0)
+ if (gzip && strcmp(basedir, "-") == 0)
{
fprintf(stderr,
_("%s: compression is not supported on standard output\n"),
Peter Eisentraut <peter_e@gmx.net> writes:
Quick patch for verification. I chose the naming -z/--gzip to mirror
GNU tar.
I would argue that -Z ought to turn on "gzip" without my having to write
-z as well (at least when the argument is greater than zero; possibly
-Z0 should be allowed as meaning "no compression").
Other than that (and the ensuing docs and help changes), looks fine.
regards, tom lane
On tis, 2011-05-24 at 15:34 -0400, Tom Lane wrote:
I would argue that -Z ought to turn on "gzip" without my having to
write
-z as well (at least when the argument is greater than zero; possibly
-Z0 should be allowed as meaning "no compression").
My concern with that is that if we ever add another compression method,
would we then add another option to control the compression level of
that method?
Peter Eisentraut <peter_e@gmx.net> writes:
On tis, 2011-05-24 at 15:34 -0400, Tom Lane wrote:
I would argue that -Z ought to turn on "gzip" without my having to write
-z as well (at least when the argument is greater than zero; possibly
-Z0 should be allowed as meaning "no compression").
My concern with that is that if we ever add another compression method,
would we then add another option to control the compression level of
that method?
Um ... what's your point? Forcing the user to type two switches instead
of one isn't going to make that hypothetical future extension any
easier, AFAICS.
But if you want to take such an extension into account right now, maybe
we ought to design that feature now. What are you seeing it as looking
like?
My thought is that "-z" should just mean "give me compression; a good
default compression setting is fine". "-Zn" could mean "I want gzip
with exactly this compression level" (thus making the presence or
absence of -z moot). If you want to specify some other compression
method altogether, use something like --lzma=N. It seems unlikely to me
that somebody who wants to override the default compression method
wouldn't want to pick the settings for it too.
regards, tom lane
On tor, 2011-05-26 at 16:54 -0400, Tom Lane wrote:
But if you want to take such an extension into account right now,
maybe we ought to design that feature now. What are you seeing it as
looking like?My thought is that "-z" should just mean "give me compression; a good
default compression setting is fine". "-Zn" could mean "I want gzip
with exactly this compression level" (thus making the presence or
absence of -z moot). If you want to specify some other compression
method altogether, use something like --lzma=N. It seems unlikely to
me that somebody who wants to override the default compression method
wouldn't want to pick the settings for it too.
I think of pg_basebackup as analogous to tar. tar has a bunch of
options to set a compression method (-Z, -z, -j, -J), but no support for
setting compression specific options. So in that sense that contradicts
your suspicion.
Peter Eisentraut <peter_e@gmx.net> writes:
On tor, 2011-05-26 at 16:54 -0400, Tom Lane wrote:
But if you want to take such an extension into account right now,
maybe we ought to design that feature now. What are you seeing it as
looking like?My thought is that "-z" should just mean "give me compression; a good
default compression setting is fine". "-Zn" could mean "I want gzip
with exactly this compression level" (thus making the presence or
absence of -z moot). If you want to specify some other compression
method altogether, use something like --lzma=N. It seems unlikely to
me that somebody who wants to override the default compression method
wouldn't want to pick the settings for it too.
I think of pg_basebackup as analogous to tar. tar has a bunch of
options to set a compression method (-Z, -z, -j, -J), but no support for
setting compression specific options. So in that sense that contradicts
your suspicion.
I would think we'd be more concerned about preserving an analogy to
pg_dump, which most certainly does expose compression-quality options.
regards, tom lane