pg_dump end comment

Started by Christopher Kings-Lynnealmost 22 years ago10 messages

chriskl@familyhealth.com.au

almost 22 years ago

This might seem a bit silly, but is there any chance we could add a
comment at the end of pg_dump text output that says '-- End of dump'?

Would make it useful for checking that you actually have a complete dump...

Chris

Tom Lane

tgl@sss.pgh.pa.us

almost 22 years ago

In reply to: Christopher Kings-Lynne (#1)

Re: pg_dump end comment

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

This might seem a bit silly, but is there any chance we could add a
comment at the end of pg_dump text output that says '-- End of dump'?

Sure --- while you're at it, put a "beginning of dump" at the start.

Is it worth adding the database name and/or other command-line
parameters given to pg_dump?

One thing to think about is the difference between a bare pg_dump and a
pg_dump/pg_restore sequence. Should these always generate identical
text output? (They do as of CVS tip, I believe, though this was not
always true before.)

Possibly this is all gilding the lily though...

regards, tom lane

Bruce Momjian

pgman@candle.pha.pa.us

almost 22 years ago

In reply to: Tom Lane (#2)

Re: pg_dump end comment

Tom Lane wrote:

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

This might seem a bit silly, but is there any chance we could add a
comment at the end of pg_dump text output that says '-- End of dump'?

Sure --- while you're at it, put a "beginning of dump" at the start.

Is it worth adding the database name and/or other command-line
parameters given to pg_dump?

One thing to think about is the difference between a bare pg_dump and a
pg_dump/pg_restore sequence. Should these always generate identical
text output? (They do as of CVS tip, I believe, though this was not
always true before.)

Possibly this is all gilding the lily though...

I like an end-of-dump marker for folks who want to check if the dump got
truncated somehow. I can see how to do that for text dumps, but what
about for tar or custom dumps?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Gavin Sherry

swm@linuxworld.com.au

almost 22 years ago

In reply to: Bruce Momjian (#3)

Re: pg_dump end comment

On Tue, 30 Mar 2004, Bruce Momjian wrote:

Tom Lane wrote:

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

This might seem a bit silly, but is there any chance we could add a
comment at the end of pg_dump text output that says '-- End of dump'?

Sure --- while you're at it, put a "beginning of dump" at the start.

Is it worth adding the database name and/or other command-line
parameters given to pg_dump?

One thing to think about is the difference between a bare pg_dump and a
pg_dump/pg_restore sequence. Should these always generate identical
text output? (They do as of CVS tip, I believe, though this was not
always true before.)

Possibly this is all gilding the lily though...

I like an end-of-dump marker for folks who want to check if the dump got
truncated somehow. I can see how to do that for text dumps, but what
about for tar or custom dumps?

Wouldn't it be more effective to test for non zero return status as this
handles -Fc cases, etc, which would be non-trivial to test.

Gavin

Bruce Momjian

pgman@candle.pha.pa.us

almost 22 years ago

In reply to: Gavin Sherry (#4)

Re: pg_dump end comment

Gavin Sherry wrote:

On Tue, 30 Mar 2004, Bruce Momjian wrote:

Tom Lane wrote:

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

This might seem a bit silly, but is there any chance we could add a
comment at the end of pg_dump text output that says '-- End of dump'?

Sure --- while you're at it, put a "beginning of dump" at the start.

Is it worth adding the database name and/or other command-line
parameters given to pg_dump?

One thing to think about is the difference between a bare pg_dump and a
pg_dump/pg_restore sequence. Should these always generate identical
text output? (They do as of CVS tip, I believe, though this was not
always true before.)

Possibly this is all gilding the lily though...

I like an end-of-dump marker for folks who want to check if the dump got
truncated somehow. I can see how to do that for text dumps, but what
about for tar or custom dumps?

Wouldn't it be more effective to test for non zero return status as this
handles -Fc cases, etc, which would be non-trivial to test.

Yes, they have to check for a proper exit from pg_dump, but there is
still a file sitting around after the dump, with no way to tell if it is
accurate.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tom Lane

tgl@sss.pgh.pa.us

almost 22 years ago

In reply to: Gavin Sherry (#4)

Re: pg_dump end comment

Gavin Sherry <swm@linuxworld.com.au> writes:

On Tue, 30 Mar 2004, Bruce Momjian wrote:

I like an end-of-dump marker for folks who want to check if the dump got
truncated somehow. I can see how to do that for text dumps, but what
about for tar or custom dumps?

Wouldn't it be more effective to test for non zero return status as this
handles -Fc cases, etc, which would be non-trivial to test.

In the tar and custom archive formats, it's fairly obvious if the file
was truncated: you'll have table-of-contents entries that point past
the end of file. The only case that really needs an endmarker is
textual scripts --- both straight from pg_dump, and produced by
pg_restore.

regards, tom lane

Philip Warner

pjw@rhyme.com.au

almost 22 years ago

In reply to: Bruce Momjian (#5)

Re: pg_dump end comment

At 12:13 AM 31/03/2004, Bruce Momjian wrote:

Yes, they have to check for a proper exit from pg_dump, but there is
still a file sitting around after the dump, with no way to tell if it is
accurate.

Why don't we write a hash into the header or footer. Then use something like:

pg_restore --verify <dump-file>

if file integrity is the objective.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.B.N. 75 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 03 5330 3172 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp.mit.edu:11371 |/

Christopher Kings-Lynne

chriskl@familyhealth.com.au

almost 22 years ago

In reply to: Gavin Sherry (#4)

Re: pg_dump end comment

I like an end-of-dump marker for folks who want to check if the dump got
truncated somehow. I can see how to do that for text dumps, but what
about for tar or custom dumps?

Wouldn't it be more effective to test for non zero return status as this
handles -Fc cases, etc, which would be non-trivial to test.

That assumes you are there at the time of dump...

Chris

scott.marlowe

scott.marlowe@ihs.com

almost 22 years ago

In reply to: Philip Warner (#7)

Re: pg_dump end comment

On Wed, 31 Mar 2004, Philip Warner wrote:

At 12:13 AM 31/03/2004, Bruce Momjian wrote:

Yes, they have to check for a proper exit from pg_dump, but there is
still a file sitting around after the dump, with no way to tell if it is
accurate.

Why don't we write a hash into the header or footer. Then use something like:

pg_restore --verify <dump-file>

if file integrity is the objective.

I like this idea. Nice to be able to check the md5 sig to make sure a
backup is not corrupted OR short.

#10

Bruce Momjian

pgman@candle.pha.pa.us

over 21 years ago

In reply to: Tom Lane (#2)

1 attachment(s)

Re: [HACKERS] pg_dump end comment

Tom Lane wrote:

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

This might seem a bit silly, but is there any chance we could add a
comment at the end of pg_dump text output that says '-- End of dump'?

Sure --- while you're at it, put a "beginning of dump" at the start.

Is it worth adding the database name and/or other command-line
parameters given to pg_dump?

One thing to think about is the difference between a bare pg_dump and a
pg_dump/pg_restore sequence. Should these always generate identical
text output? (They do as of CVS tip, I believe, though this was not
always true before.)

I have added code to print end-of-dump markers for pg_dump and
pg_dumpall:

--
-- PostgreSQL database dump complete
--

--
-- PostgreSQL database cluster dump complete
--

This mirrors the markers used at the start of the files.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Attachments:

/bjm/difftext/plainDownload

Index: src/bin/pg_dump/pg_backup_archiver.c
===================================================================
RCS file: /cvsroot/pgsql-server/src/bin/pg_dump/pg_backup_archiver.c,v
retrieving revision 1.86
diff -c -c -r1.86 pg_backup_archiver.c
*** src/bin/pg_dump/pg_backup_archiver.c	22 Apr 2004 02:39:10 -0000	1.86
--- src/bin/pg_dump/pg_backup_archiver.c	19 May 2004 21:17:43 -0000
***************
*** 384,389 ****
--- 384,391 ----
  			AH->blobConnection = NULL;
  		}
  	}
+ 
+ 	ahprintf(AH, "--\n-- PostgreSQL database dump complete\n--\n\n");
  }
  
  /*
Index: src/bin/pg_dump/pg_dumpall.c
===================================================================
RCS file: /cvsroot/pgsql-server/src/bin/pg_dump/pg_dumpall.c,v
retrieving revision 1.33
diff -c -c -r1.33 pg_dumpall.c
*** src/bin/pg_dump/pg_dumpall.c	14 May 2004 17:04:47 -0000	1.33
--- src/bin/pg_dump/pg_dumpall.c	19 May 2004 21:17:43 -0000
***************
*** 242,247 ****
--- 242,252 ----
  	}
  
  	PQfinish(conn);
+ 
+ 	printf("--\n");
+ 	printf("-- PostgreSQL database cluster dump complete\n");
+ 	printf("--\n\n");
+ 
  	exit(0);
  }