Display of timestamp in pg_dump custom format

Started by Bruce Momjianover 11 years ago10 messages
#1Bruce Momjian
bruce@momjian.us

The table of contents for pg_restore -l shows the time the archive was
made as local time (it uses ctime()):

; Archive created at Wed Apr 30 10:03:28 2014

Is this clear enough that it is local time? Should we display this
better, perhaps with a time zone designation?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#2Gavin Flower
GavinFlower@archidevsys.co.nz
In reply to: Bruce Momjian (#1)
Re: Display of timestamp in pg_dump custom format

On 01/05/14 02:51, Bruce Momjian wrote:

The table of contents for pg_restore -l shows the time the archive was
made as local time (it uses ctime()):

; Archive created at Wed Apr 30 10:03:28 2014

Is this clear enough that it is local time? Should we display this
better, perhaps with a time zone designation?

I think it would be good to include the time zone, as we are all very
international these days - and in Australia, adjacent states have
different dates for the summer time transition!

Personally, I would like to see the date in the format 2014-04-30, but
having the day of the week is good.

Milliseconds might be useful, if you want to check logs files.

Cheers,
Gavin

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Bruce Momjian
bruce@momjian.us
In reply to: Gavin Flower (#2)
Re: Display of timestamp in pg_dump custom format

On Thu, May 1, 2014 at 08:27:49AM +1200, Gavin Flower wrote:

On 01/05/14 02:51, Bruce Momjian wrote:

The table of contents for pg_restore -l shows the time the archive was
made as local time (it uses ctime()):

; Archive created at Wed Apr 30 10:03:28 2014

Is this clear enough that it is local time? Should we display this
better, perhaps with a time zone designation?

I think it would be good to include the time zone, as we are all
very international these days - and in Australia, adjacent states
have different dates for the summer time transition!

Personally, I would like to see the date in the format 2014-04-30,
but having the day of the week is good.

Milliseconds might be useful, if you want to check logs files.

OK, I will work on it for 9.5. Thanks.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Gavin Flower
GavinFlower@archidevsys.co.nz
In reply to: Bruce Momjian (#3)
Re: Display of timestamp in pg_dump custom format

On 01/05/14 12:04, Bruce Momjian wrote:

On Thu, May 1, 2014 at 08:27:49AM +1200, Gavin Flower wrote:

On 01/05/14 02:51, Bruce Momjian wrote:

The table of contents for pg_restore -l shows the time the archive was
made as local time (it uses ctime()):

; Archive created at Wed Apr 30 10:03:28 2014

Is this clear enough that it is local time? Should we display this
better, perhaps with a time zone designation?

I think it would be good to include the time zone, as we are all
very international these days - and in Australia, adjacent states
have different dates for the summer time transition!

Personally, I would like to see the date in the format 2014-04-30,
but having the day of the week is good.

Milliseconds might be useful, if you want to check logs files.

OK, I will work on it for 9.5. Thanks.

So the it would then read something like:

; Archive created at Wed 2014-04-30 10:03:28.042 NZST

(but with the correct appropriate time zone designation)?

Cheers,
Gavin

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Bruce Momjian
bruce@momjian.us
In reply to: Gavin Flower (#4)
Re: Display of timestamp in pg_dump custom format

On Thu, May 1, 2014 at 12:33:51PM +1200, Gavin Flower wrote:

On 01/05/14 12:04, Bruce Momjian wrote:

On Thu, May 1, 2014 at 08:27:49AM +1200, Gavin Flower wrote:

On 01/05/14 02:51, Bruce Momjian wrote:

The table of contents for pg_restore -l shows the time the archive was
made as local time (it uses ctime()):

; Archive created at Wed Apr 30 10:03:28 2014

Is this clear enough that it is local time? Should we display this
better, perhaps with a time zone designation?

I think it would be good to include the time zone, as we are all
very international these days - and in Australia, adjacent states
have different dates for the summer time transition!

Personally, I would like to see the date in the format 2014-04-30,
but having the day of the week is good.

Milliseconds might be useful, if you want to check logs files.

OK, I will work on it for 9.5. Thanks.

So the it would then read something like:

; Archive created at Wed 2014-04-30 10:03:28.042 NZST

(but with the correct appropriate time zone designation)?

I think we would use a numeric offset.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#5)
1 attachment(s)
Re: Display of timestamp in pg_dump custom format

On Thu, May 1, 2014 at 12:09:34PM -0400, Bruce Momjian wrote:

On Thu, May 1, 2014 at 12:33:51PM +1200, Gavin Flower wrote:

On 01/05/14 12:04, Bruce Momjian wrote:

On Thu, May 1, 2014 at 08:27:49AM +1200, Gavin Flower wrote:

On 01/05/14 02:51, Bruce Momjian wrote:

The table of contents for pg_restore -l shows the time the archive was
made as local time (it uses ctime()):

; Archive created at Wed Apr 30 10:03:28 2014

Is this clear enough that it is local time? Should we display this
better, perhaps with a time zone designation?

I think it would be good to include the time zone, as we are all
very international these days - and in Australia, adjacent states
have different dates for the summer time transition!

Personally, I would like to see the date in the format 2014-04-30,
but having the day of the week is good.

Milliseconds might be useful, if you want to check logs files.

OK, I will work on it for 9.5. Thanks.

So the it would then read something like:

; Archive created at Wed 2014-04-30 10:03:28.042 NZST

(but with the correct appropriate time zone designation)?

I think we would use a numeric offset.

I ended up going with the string-based timezone as I was worried that
the sign of the timezone could easily confuse people because the SQL
timezone offset sign is often different from the OS timezone. The new
output is:

;
; Archive created at Wed Sep 3 16:12:21 2014 EST <--
; dbname: test
; TOC Entries: 8
; Compression: -1
; Dump Version: 1.12-0
; Format: CUSTOM
; Integer: 4 bytes
; Offset: 8 bytes
; Dumped from database version: 9.5devel
; Dumped by pg_dump version: 9.5devel

Patch attached.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

Attachments:

dumpstamp.difftext/x-diff; charset=us-asciiDownload
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
new file mode 100644
index 0018720..4296c11
*** a/src/bin/pg_dump/pg_backup_archiver.c
--- b/src/bin/pg_dump/pg_backup_archiver.c
*************** PrintTOCSummary(Archive *AHX, RestoreOpt
*** 969,975 ****
  	if (ropt->filename)
  		SetOutput(AH, ropt->filename, 0 /* no compression */ );
  
! 	ahprintf(AH, ";\n; Archive created at %s", ctime(&AH->createDate));
  	ahprintf(AH, ";     dbname: %s\n;     TOC Entries: %d\n;     Compression: %d\n",
  			 AH->archdbname, AH->tocCount, AH->compression);
  
--- 969,975 ----
  	if (ropt->filename)
  		SetOutput(AH, ropt->filename, 0 /* no compression */ );
  
! 	ahprintf(AH, ";\n; Archive created at %.24s %s\n", ctime(&AH->createDate), *tzname);
  	ahprintf(AH, ";     dbname: %s\n;     TOC Entries: %d\n;     Compression: %d\n",
  			 AH->archdbname, AH->tocCount, AH->compression);
  
#7Gavin Flower
GavinFlower@archidevsys.co.nz
In reply to: Bruce Momjian (#6)
Re: Display of timestamp in pg_dump custom format

On 04/09/14 08:13, Bruce Momjian wrote:

On Thu, May 1, 2014 at 12:09:34PM -0400, Bruce Momjian wrote:

On Thu, May 1, 2014 at 12:33:51PM +1200, Gavin Flower wrote:

On 01/05/14 12:04, Bruce Momjian wrote:

On Thu, May 1, 2014 at 08:27:49AM +1200, Gavin Flower wrote:

On 01/05/14 02:51, Bruce Momjian wrote:

The table of contents for pg_restore -l shows the time the archive was
made as local time (it uses ctime()):

; Archive created at Wed Apr 30 10:03:28 2014

Is this clear enough that it is local time? Should we display this
better, perhaps with a time zone designation?

I think it would be good to include the time zone, as we are all
very international these days - and in Australia, adjacent states
have different dates for the summer time transition!

Personally, I would like to see the date in the format 2014-04-30,
but having the day of the week is good.

Milliseconds might be useful, if you want to check logs files.

OK, I will work on it for 9.5. Thanks.

So the it would then read something like:

; Archive created at Wed 2014-04-30 10:03:28.042 NZST

(but with the correct appropriate time zone designation)?

I think we would use a numeric offset.

I ended up going with the string-based timezone as I was worried that
the sign of the timezone could easily confuse people because the SQL
timezone offset sign is often different from the OS timezone. The new
output is:

;
; Archive created at Wed Sep 3 16:12:21 2014 EST <--
; dbname: test
; TOC Entries: 8
; Compression: -1
; Dump Version: 1.12-0
; Format: CUSTOM
; Integer: 4 bytes
; Offset: 8 bytes
; Dumped from database version: 9.5devel
; Dumped by pg_dump version: 9.5devel

Patch attached.

I would prefer the date in a sane numeric format to the left of the time
(similar to what I suggested above), easier to sort (if a sort is
required) - it is also easier to use regular expressions to select
statement in an arbitrary date/time range.

I don't always know in advance that I need to debug something, so I tend
to try and ensure that the relevant data is easy to find, even when I
currently don't expect ever to do so. This is a lesson that I have
learnt from over 40 years of commercial programming experience using a
variety of languages on a wide range of platforms.

Most likely, I will never need to worry about the precise format of
Archive statement output, but ...

Cheers,
Gavin

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Bruce Momjian
bruce@momjian.us
In reply to: Gavin Flower (#7)
Re: Display of timestamp in pg_dump custom format

On Thu, Sep 4, 2014 at 12:02:19PM +1200, Gavin Flower wrote:

I would prefer the date in a sane numeric format to the left of the
time (similar to what I suggested above), easier to sort (if a sort
is required) - it is also easier to use regular expressions to
select statement in an arbitrary date/time range.

I don't always know in advance that I need to debug something, so I
tend to try and ensure that the relevant data is easy to find, even
when I currently don't expect ever to do so. This is a lesson that
I have learnt from over 40 years of commercial programming
experience using a variety of languages on a wide range of
platforms.

Most likely, I will never need to worry about the precise format of
Archive statement output, but ...

I can't seem to find a way to get the timezone offset via C; see:

http://stackoverflow.com/questions/635780/why-does-glibc-timezone-global-not-agree-with-system-time-on-dst

On Linux, do 'man timezone' for details. 'timezone' has the non-DST
offset from GMT, and 'daylight' is a boolean which indicates DST, but
not how much time is different for DST, and I am not sure it is always
an hour. In fact 'daylight' is documented as saying whether there is
every a daylight savings time, not that DST is active.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#8)
1 attachment(s)
Re: Display of timestamp in pg_dump custom format

On Wed, Sep 3, 2014 at 08:33:31PM -0400, Bruce Momjian wrote:

I can't seem to find a way to get the timezone offset via C; see:

http://stackoverflow.com/questions/635780/why-does-glibc-timezone-global-not-agree-with-system-time-on-dst

On Linux, do 'man timezone' for details. 'timezone' has the non-DST
offset from GMT, and 'daylight' is a boolean which indicates DST, but
not how much time is different for DST, and I am not sure it is always
an hour. In fact 'daylight' is documented as saying whether there is
every a daylight savings time, not that DST is active.

Uh, not sure what I was thinking --- strftime() is the way to go. Here
is the new output:

;
; Archive created at 2014-09-04 13:00:15 -0400 <---
; dbname: test
; TOC Entries: 8
; Compression: -1
; Dump Version: 1.12-0
; Format: CUSTOM
; Integer: 4 bytes
; Offset: 8 bytes
; Dumped from database version: 9.5devel
; Dumped by pg_dump version: 9.5devel

I found two other places in our dump code that use strftime with a
similar format, but they had problems with the timezone string on
Windows, so I switched those over to use a numeric timezone offset as
well.

Patch attached.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

Attachments:

dumpstamp.difftext/x-diff; charset=us-asciiDownload
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
new file mode 100644
index 0018720..ded9135
*** a/src/bin/pg_dump/pg_backup_archiver.c
--- b/src/bin/pg_dump/pg_backup_archiver.c
*************** PrintTOCSummary(Archive *AHX, RestoreOpt
*** 964,975 ****
  	teSection	curSection;
  	OutputContext sav;
  	const char *fmtName;
  
  	sav = SaveOutput(AH);
  	if (ropt->filename)
  		SetOutput(AH, ropt->filename, 0 /* no compression */ );
  
! 	ahprintf(AH, ";\n; Archive created at %s", ctime(&AH->createDate));
  	ahprintf(AH, ";     dbname: %s\n;     TOC Entries: %d\n;     Compression: %d\n",
  			 AH->archdbname, AH->tocCount, AH->compression);
  
--- 964,978 ----
  	teSection	curSection;
  	OutputContext sav;
  	const char *fmtName;
+ 	struct tm  *tm = localtime(&AH->createDate);
+ 	char		stamp_str[64];
  
  	sav = SaveOutput(AH);
  	if (ropt->filename)
  		SetOutput(AH, ropt->filename, 0 /* no compression */ );
  
! 	strftime(stamp_str, sizeof(stamp_str), "%Y-%m-%d %H:%M:%S %z", tm);
! 	ahprintf(AH, ";\n; Archive created at %s\n", stamp_str);
  	ahprintf(AH, ";     dbname: %s\n;     TOC Entries: %d\n;     Compression: %d\n",
  			 AH->archdbname, AH->tocCount, AH->compression);
  
*************** checkSeek(FILE *fp)
*** 3455,3475 ****
  static void
  dumpTimestamp(ArchiveHandle *AH, const char *msg, time_t tim)
  {
! 	char		buf[256];
  
! 	/*
! 	 * We don't print the timezone on Win32, because the names are long and
! 	 * localized, which means they may contain characters in various random
! 	 * encodings; this has been seen to cause encoding errors when reading the
! 	 * dump script.
! 	 */
! 	if (strftime(buf, sizeof(buf),
! #ifndef WIN32
! 				 "%Y-%m-%d %H:%M:%S %Z",
! #else
! 				 "%Y-%m-%d %H:%M:%S",
! #endif
! 				 localtime(&tim)) != 0)
  		ahprintf(AH, "-- %s %s\n\n", msg, buf);
  }
  
--- 3458,3466 ----
  static void
  dumpTimestamp(ArchiveHandle *AH, const char *msg, time_t tim)
  {
! 	char		buf[64];
  
! 	if (strftime(buf, sizeof(buf), "%Y-%m-%d %H:%M:%S %z", localtime(&tim)) != 0)
  		ahprintf(AH, "-- %s %s\n\n", msg, buf);
  }
  
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
new file mode 100644
index 4050091..b2b3e6f
*** a/src/bin/pg_dump/pg_dumpall.c
--- b/src/bin/pg_dump/pg_dumpall.c
*************** executeCommand(PGconn *conn, const char
*** 2039,2060 ****
  static void
  dumpTimestamp(char *msg)
  {
! 	char		buf[256];
  	time_t		now = time(NULL);
  
! 	/*
! 	 * We don't print the timezone on Win32, because the names are long and
! 	 * localized, which means they may contain characters in various random
! 	 * encodings; this has been seen to cause encoding errors when reading the
! 	 * dump script.
! 	 */
! 	if (strftime(buf, sizeof(buf),
! #ifndef WIN32
! 				 "%Y-%m-%d %H:%M:%S %Z",
! #else
! 				 "%Y-%m-%d %H:%M:%S",
! #endif
! 				 localtime(&now)) != 0)
  		fprintf(OPF, "-- %s %s\n\n", msg, buf);
  }
  
--- 2039,2048 ----
  static void
  dumpTimestamp(char *msg)
  {
! 	char		buf[64];
  	time_t		now = time(NULL);
  
! 	if (strftime(buf, sizeof(buf), "%Y-%m-%d %H:%M:%S %z", localtime(&now)) != 0)
  		fprintf(OPF, "-- %s %s\n\n", msg, buf);
  }
  
#10Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#9)
Re: Display of timestamp in pg_dump custom format

On Thu, Sep 4, 2014 at 01:19:31PM -0400, Bruce Momjian wrote:

Uh, not sure what I was thinking --- strftime() is the way to go. Here
is the new output:

;
; Archive created at 2014-09-04 13:00:15 -0400 <---
; dbname: test
; TOC Entries: 8
; Compression: -1
; Dump Version: 1.12-0
; Format: CUSTOM
; Integer: 4 bytes
; Offset: 8 bytes
; Dumped from database version: 9.5devel
; Dumped by pg_dump version: 9.5devel

I found two other places in our dump code that use strftime with a
similar format, but they had problems with the timezone string on
Windows, so I switched those over to use a numeric timezone offset as
well.

Patch applied.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers