pg_dump compress

Started by Roger Niederlandover 14 years ago15 messagesgeneral
Jump to latest
#1Roger Niederland
roger@niederland.com

Using pg_dump from the command line with the exe included in windows 32
bit install in postgresql 9.1.
pg_dump -U username -f somefile.sql.gz -Z 9 -i dbname

outputs a file that is in plain text. In previous versions of
postgresql, this output a gzipped file.
Also tried: pg_dump -U username -Z 9 -i dbname > somefile.sql.gz
got the same results.

Tried to revert to a previous version of pg_dump from pgadmin 1.12.
The dump is aborted because of server version mismatch (server version
9.1.0; pg_dump version: 9.0.1)

Looked at the pg_dump documentation and it was not noted that the -Z
option changed.

Roger

In reply to: Roger Niederland (#1)
Re: pg_dump compress

On Thu, Sep 22, 2011 at 11:00:10PM -0700, Roger Niederland wrote:

Using pg_dump from the command line with the exe included in windows
32 bit install in postgresql 9.1.
pg_dump -U username -f somefile.sql.gz -Z 9 -i dbname

please note that pg_dump --help shows:
-Z, --compress=0-9 compression level for compressed formats

hint: the part "for compressed formats" is critical.

plain dump is not compressed, you'd have to use -Fc to get compression,
but it's good to use anyway.

Best regards,

depesz

#3Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Roger Niederland (#1)
Re: pg_dump compress

On Thursday, September 22, 2011 11:00:10 pm Roger Niederland wrote:

Using pg_dump from the command line with the exe included in windows 32
bit install in postgresql 9.1.
pg_dump -U username -f somefile.sql.gz -Z 9 -i dbname

outputs a file that is in plain text. In previous versions of
postgresql, this output a gzipped file.
Also tried: pg_dump -U username -Z 9 -i dbname > somefile.sql.gz
got the same results.

Tried to revert to a previous version of pg_dump from pgadmin 1.12.
The dump is aborted because of server version mismatch (server version
9.1.0; pg_dump version: 9.0.1)

Looked at the pg_dump documentation and it was not noted that the -Z
option changed.

Works here on a Linux machine. Maybe the gzip code is not installed in your
Windows install? By the way as concerns the -i option, from the docs:

http://www.postgresql.org/docs/9.1/interactive/app-pgdump.html

-i
--ignore-version

A deprecated option that is now ignored.

So ignore is ignored:)

Roger

--
Adrian Klaver
adrian.klaver@gmail.com

#4Adrian Klaver
adrian.klaver@aklaver.com
In reply to: hubert depesz lubaczewski (#2)
Re: pg_dump compress

On Friday, September 23, 2011 6:46:49 am hubert depesz lubaczewski wrote:

On Thu, Sep 22, 2011 at 11:00:10PM -0700, Roger Niederland wrote:

Using pg_dump from the command line with the exe included in windows
32 bit install in postgresql 9.1.
pg_dump -U username -f somefile.sql.gz -Z 9 -i dbname

please note that pg_dump --help shows:
-Z, --compress=0-9 compression level for compressed formats

hint: the part "for compressed formats" is critical.

The docs show:

http://www.postgresql.org/docs/9.1/interactive/app-pgdump.html

-Z 0..9
--compress=0..9

Specify the compression level to use. Zero means no compression. For the
custom archive format, this specifies compression of individual table-data
segments, and the default is to compress at a moderate level. For plain text
output, setting a nonzero compression level causes the entire output file to be
compressed, as though it had been fed through gzip; but the default is not to
compress. The tar archive format currently does not support compression at all.

plain dump is not compressed, you'd have to use -Fc to get compression,
but it's good to use anyway.

Best regards,

depesz

--
Adrian Klaver
adrian.klaver@gmail.com

#5Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Roger Niederland (#1)
Re: pg_dump compress

On Thursday, September 22, 2011 11:00:10 pm Roger Niederland wrote:

Using pg_dump from the command line with the exe included in windows 32
bit install in postgresql 9.1.
pg_dump -U username -f somefile.sql.gz -Z 9 -i dbname

Ignore my previous post. I just realized that I was using the 9.0.4 version of
pg_dump not 9.1. Sorry for the noise.

Roger

--
Adrian Klaver
adrian.klaver@gmail.com

#6Roger Niederland
roger@niederland.com
In reply to: hubert depesz lubaczewski (#2)
Re: pg_dump compress

On 9/23/2011 6:46 AM, hubert depesz lubaczewski wrote:

On Thu, Sep 22, 2011 at 11:00:10PM -0700, Roger Niederland wrote:

Using pg_dump from the command line with the exe included in windows
32 bit install in postgresql 9.1.
pg_dump -U username -f somefile.sql.gz -Z 9 -i dbname

please note that pg_dump --help shows:
-Z, --compress=0-9 compression level for compressed formats

hint: the part "for compressed formats" is critical.

plain dump is not compressed, you'd have to use -Fc to get compression,
but it's good to use anyway.

Best regards,

depesz

On previous versions the -Z 9 option would compress the plain text sql
output to a gzipped file.
Now it now longer compresses. Using the -Fc the outputs a compressed
file, which is only readable
by pg_restore.

Roger

#7Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Roger Niederland (#1)
Re: pg_dump compress

On Thursday, September 22, 2011 11:00:10 pm Roger Niederland wrote:

Using pg_dump from the command line with the exe included in windows 32
bit install in postgresql 9.1.
pg_dump -U username -f somefile.sql.gz -Z 9 -i dbname

outputs a file that is in plain text. In previous versions of
postgresql, this output a gzipped file.
Also tried: pg_dump -U username -Z 9 -i dbname > somefile.sql.gz
got the same results.

Tried to revert to a previous version of pg_dump from pgadmin 1.12.
The dump is aborted because of server version mismatch (server version
9.1.0; pg_dump version: 9.0.1)

Looked at the pg_dump documentation and it was not noted that the -Z
option changed.

I set up a 9.1 installation on my Linux machine and can confirm that the 9.1
version of pg_dump does not compress plain text format dump files.

Roger

--
Adrian Klaver
adrian.klaver@gmail.com

#8Grzegorz Jaśkiewicz
gryzman@gmail.com
In reply to: Roger Niederland (#6)
Re: pg_dump compress

can you pipe things on windows ?
It's a desktop system after all, but dos had that sort of a feature -
I seem to remember.

#9Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Roger Niederland (#6)
Re: pg_dump compress

On Friday, September 23, 2011 7:26:19 am Roger Niederland wrote:

On 9/23/2011 6:46 AM, hubert depesz lubaczewski wrote:

On Thu, Sep 22, 2011 at 11:00:10PM -0700, Roger Niederland wrote:

Using pg_dump from the command line with the exe included in windows
32 bit install in postgresql 9.1.
pg_dump -U username -f somefile.sql.gz -Z 9 -i dbname

please note that pg_dump --help shows:
-Z, --compress=0-9 compression level for compressed formats

hint: the part "for compressed formats" is critical.

plain dump is not compressed, you'd have to use -Fc to get compression,
but it's good to use anyway.

Best regards,

depesz

On previous versions the -Z 9 option would compress the plain text sql
output to a gzipped file.
Now it now longer compresses. Using the -Fc the outputs a compressed
file, which is only readable
by pg_restore.

Seems there has been a code change in pg_dump.c that ignores compression level
for plain text.

From line 537:

/*
* Ignore compression level for plain format. XXX: This is a bit
* inconsistent, tar-format throws an error instead.
*/

The documentation has not caught up with this change.

Roger

--
Adrian Klaver
adrian.klaver@gmail.com

#10Filip Rembiałkowski
plk.zuber@gmail.com
In reply to: Grzegorz Jaśkiewicz (#8)
Re: pg_dump compress

2011/9/23 Gregg Jaskiewicz <gryzman@gmail.com>

can you pipe things on windows ?

Yes you can. It surprised me positively several years ago.
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/redirection.mspx?mfr=true

It's a desktop system after all,

:-) what a nice dose of condescending irony.
You know, they are selling a "server OS" on top of that. Quite a lot of
copies.

cheers, Filip

#11Grzegorz Jaśkiewicz
gryzman@gmail.com
In reply to: Filip Rembiałkowski (#10)
Re: pg_dump compress

Oh, neat.
And I'll call myself wizard. People will think I am one...

#12Roger Niederland
roger@niederland.com
In reply to: Adrian Klaver (#9)
Re: pg_dump compress

On 9/23/2011 5:18 PM, Adrian Klaver wrote:

On Friday, September 23, 2011 7:26:19 am Roger Niederland wrote:

On 9/23/2011 6:46 AM, hubert depesz lubaczewski wrote:

On Thu, Sep 22, 2011 at 11:00:10PM -0700, Roger Niederland wrote:

Using pg_dump from the command line with the exe included in windows
32 bit install in postgresql 9.1.
pg_dump -U username -f somefile.sql.gz -Z 9 -i dbname

please note that pg_dump --help shows:
-Z, --compress=0-9 compression level for compressed formats

hint: the part "for compressed formats" is critical.

plain dump is not compressed, you'd have to use -Fc to get compression,
but it's good to use anyway.

Best regards,

depesz

On previous versions the -Z 9 option would compress the plain text sql
output to a gzipped file.
Now it now longer compresses. Using the -Fc the outputs a compressed
file, which is only readable
by pg_restore.

Seems there has been a code change in pg_dump.c that ignores compression level
for plain text.

From line 537:

/*
* Ignore compression level for plain format. XXX: This is a bit
* inconsistent, tar-format throws an error instead.
*/

The documentation has not caught up with this change.

Roger

Just did not expect that the compression would be removed for plain files.
In case anyone cares... 7-zip.org provides a 32 bit and 64 bit program
which can utilize windows pipes.
Probably others available, but this one works for me.

Thanks,
Roger

#13Marti Raudsepp
marti@juffo.org
In reply to: Roger Niederland (#12)
Re: pg_dump compress

On Sat, Sep 24, 2011 at 17:16, Roger Niederland <roger@niederland.com> wrote:

Just did not expect that the compression would be removed for plain files.

Agreed, I'd say this is a regression that I would like to see
addressed in PostgreSQL 9.1.1. I'm sure you won't be the only to be
surprised by this.

The fact that the change wasn't documented suggests that it was made
by accident.

Regards,
Marti

#14Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Roger Niederland (#12)
Re: pg_dump compress

On Saturday, September 24, 2011 7:16:11 am Roger Niederland wrote:

On 9/23/2011 5:18 PM, Adrian Klaver wrote:

On Friday, September 23, 2011 7:26:19 am Roger Niederland wrote:

On 9/23/2011 6:46 AM, hubert depesz lubaczewski wrote:

On Thu, Sep 22, 2011 at 11:00:10PM -0700, Roger Niederland wrote:

Using pg_dump from the command line with the exe included in windows
32 bit install in postgresql 9.1.
pg_dump -U username -f somefile.sql.gz -Z 9 -i dbname

please note that pg_dump --help shows:
-Z, --compress=0-9 compression level for compressed
formats

hint: the part "for compressed formats" is critical.

plain dump is not compressed, you'd have to use -Fc to get compression,
but it's good to use anyway.

Best regards,

depesz

On previous versions the -Z 9 option would compress the plain text sql
output to a gzipped file.
Now it now longer compresses. Using the -Fc the outputs a compressed
file, which is only readable
by pg_restore.

Seems there has been a code change in pg_dump.c that ignores compression
level for plain text.

From line 537:
/*

* Ignore compression level for plain format. XXX: This is a bit
* inconsistent, tar-format throws an error instead.

*/

The documentation has not caught up with this change.

Roger

Just did not expect that the compression would be removed for plain files.
In case anyone cares... 7-zip.org provides a 32 bit and 64 bit program
which can utilize windows pipes.

Not sure why the compression was removed, there is no explanation in either the
commit or the release notes. If you are interested, the commit that removed it
is:
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;f=src/bin/pg_dump/pg_dump.c;h=7f508f1c6b515df66d27f860b2faa7b5761fa55d

You could submit a bug report and see where that leads:
http://www.postgresql.org/support/submitbug

Probably others available, but this one works for me.

Thanks,
Roger

--
Adrian Klaver
adrian.klaver@gmail.com

#15Tom Lane
tgl@sss.pgh.pa.us
In reply to: Adrian Klaver (#14)
Re: pg_dump compress

Adrian Klaver <adrian.klaver@gmail.com> writes:

On Saturday, September 24, 2011 7:16:11 am Roger Niederland wrote:

[ pg_dump 9.1 no longer honors -Z when emitting plain text ]

Not sure why the compression was removed, there is no explanation in either the
commit or the release notes. If you are interested, the commit that removed it
is:
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;f=src/bin/pg_dump/pg_dump.c;h=7f508f1c6b515df66d27f860b2faa7b5761fa55d

It appears to me that this was just a careless misunderstanding of what
pg_dump is supposed to do with this switch. Heikki, do you recall any
intentional change here, and if so why wasn't it documented?

regards, tom lane