AW: AW: Modified pg_dump & new pg_restore need testing. ..
At 09:56 3/07/00 +0200, Zeugswetter Andreas SB wrote:
- the default output file format is a custom format with compressed
sections (the data dumps). It is NOT a text file.Can this be turned off, or made to be feature, that you can
turn on ?
Imho most dumps will be piped to a locally
optimized compressor (a tape, a storage manager, lzop, ...) anyway,
thus most of the time a backup would compress twice.Yes. The default is now plain text for compatibility with the original
pg_dump (and with pg_dumpall). But by going to plain text
output you gain
none of the features of pg_restore (reordering, selection etc).
Could the text format be changed in a compatible way,
that would allow pg_restore's features ? I am thinking of inserted comment
lines that describe the needed sections.
Imho the new default format does not need to be compatible
with pg_dump's output. I would still not compress by default.
Andreas
It just occurred to me that I may be missing something...do you mean you
would prefer to avoid the 'COPY' commands as the backup technique? If so,
I
think that's best left to somebody else (or me, but not now, and probably
not until the WAL is implemented).
Yes, that was what I was thinking. I don't see a conflicting area with WAL.
I would see this as an extension to libpq and how it handles binary cursors
across different platforms.
Also, defining your own output format is pretty easy; the new
code defines
a top level interface (for pg_dump and pg_restore) and an archiver
interface (for output file formats), if you really want a
different format.
Sounds like you are doing a great job :-)
My favorite defaults would probably be -Fc -Z0 as you noted in another
posting.
Andreas
Import Notes
Resolved by subject fallback
At 10:39 3/07/00 +0200, Zeugswetter Andreas SB wrote:
Could the text format be changed in a compatible way,
that would allow pg_restore's features ? I am thinking of inserted comment
lines that describe the needed sections.
Quite possibly; although part of my motivation in writing to the new format
was to avoid writing a text parser. There is no reason why it couldn't use
a text file by doing a pass through the file to construct a TOC, but this
seems like a bad idea for large backups, and not really necessary for small
backups (especially with the -Z option).
That said, I'm happy to give it a go - the output file is already pretty
'parsable'.
Imho the new default format does not need to be compatible
with pg_dump's output. I would still not compress by default.
I'm happy to do this; at the moment it uses zlib default compression by
default for the custom format, and no compression for the plain text
output. I'm happy to make uncompressed output in all cases, and require the
'-Z' qualifier before I compress.
Does anybody else have a preference on this?
----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 0500 83 82 82 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/
At 10:49 3/07/00 +0200, Zeugswetter Andreas SB wrote:
It just occurred to me that I may be missing something...do you mean you
would prefer to avoid the 'COPY' commands as the backup technique? If so,I
think that's best left to somebody else (or me, but not now, and probably
not until the WAL is implemented).Yes, that was what I was thinking. I don't see a conflicting area with WAL.
I would see this as an extension to libpq and how it handles binary cursors
across different platforms.
OK: I know nothing about this, but I'd be interested to learn.
The reason I mentioned the WAL is that I think that any more advanced
backup strategy needs to be integrated into the journaling system (ie. the
WAL). Ideally, we should be able to snapshot a consistent view of raw data
and apply a copy of the journal to this snapshot. In other systems I use
(Dec/Rdb), this is done via a page-based backup (I think) and an
after-image journal (both of which I *think* are at a lower level than
tables & rows - more like pages & slots.
As a result, I had planned to wait and see what the WAL did for backup
strategies...
But, in the mean time, if you think a binary pg_dump is possible (an
relatively easy), I'd be interested to try to integrate it...
----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 0500 83 82 82 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/