Date/time on glibc2 linux
Hello!
The following gave me wrong result:
xxx=> select datetime(current_date, '11:00');
datetime
----------------------------
Thu 03 Dec 17:00:00 1998 MSK
(1 row)
and I started investigation. Soon I found I only have the problem on
glibc2-based linuxes (RedHat 5.1 and Debian 2.0), RedHat 4.2 and Solaris
are Ok.
Well, it looked like an error in glibc2, and I continued. I wrote the
program:
-----
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
int main(int argc, char **argv) {
char strf_buf[100];
time_t today_t;
struct tm * today;
memset(strf_buf, 0, 99);
today_t = time(NULL);
today = localtime(&today_t);
strftime(strf_buf, 100, "%Z", today);
printf("%s\n", strf_buf);
/* In Moscow I expect MSK/MSD/1/10800 */
printf("%s\n%s\n%d\n%ld\n", tzname[0], tzname[1], daylight, timezone);
return 0;
}
-----
I expected to have results:
-----
MSK
MSK
MSD
1
-10800
-----
Ok, I've got it as expected on RH4.2 and Solaris. RH5.1 and Debian2.0
gave wrong results (although 1st line was Ok: MSK - produced by strftime).
Continued, I found to my big surprize that Python programs do date/time
arithmetic right an all platforms. I looked into python's timemodule.c and
found some code with ifdef's. When I looked into config.h, I found that on
"good" platforms there is HAVE_TZNAME defined, but on "bad" platforms it is
undefined. Because of this definition "bad" platforms execute the following
code:
#else /* !HAVE_TZNAME */
#if HAVE_TM_ZONE
{
#define YEAR ((time_t)((365 * 24 + 6) * 3600))
time_t t;
struct tm *p;
long winterzone, summerzone;
char wintername[10], summername[10];
/* XXX This won't work on the southern hemisphere.
XXX Anybody got a better idea? */
t = (time((time_t *)0) / YEAR) * YEAR;
p = localtime(&t);
winterzone = -p->tm_gmtoff;
strncpy(wintername, p->tm_zone ? p->tm_zone : " ", 9);
wintername[9] = '\0';
t += YEAR/2;
p = localtime(&t);
summerzone = -p->tm_gmtoff;
strncpy(summername, p->tm_zone ? p->tm_zone : " ", 9);
summername[9] = '\0';
ins(d, "timezone", PyInt_FromLong(winterzone));
ins(d, "altzone", PyInt_FromLong(summerzone));
ins(d, "daylight",
PyInt_FromLong((long)(winterzone != summerzone)));
ins(d, "tzname",
Py_BuildValue("(zz)", wintername, summername));
}
#else
WOW!!! Look, look here - how tzname and timezone calculated! (Actually,
recalculated - glibc2 already has tzname/timezone, but initialized to wrong
values.)
I am pretty sure date/time arithmetic on Postgres should be changed
accordingly. Where can I start in postgres sources? Anyone to help?
Oleg.
----
Oleg Broytmann http://members.tripod.com/~phd2/ phd2@earthling.net
Programmers don't die, they just GOSUB without RETURN.
The following gave me wrong result:
I am pretty sure date/time arithmetic on Postgres should be changed
accordingly. Where can I start in postgres sources? Anyone to help?
I see the same symptom on my RH5.1 (glibc2) box here:
lockhart=> select datetime('today'::date);
datetime
----------------------------
Fri Dec 04 06:00:00 1998 MSK
(1 row)
Whereas my RH4.2 (libc5) box gets it right:
postgres=> select datetime('today'::date);
datetime
----------------------------
Fri Dec 04 00:00:00 1998 MSK
(1 row)
This isn't specific to your timezone; I see the same problem with
PST8PST. The routine actually getting invoked is date_datetime() in
backend/utils/adt/datetime.c. I would be reluctant to scatter
glibc2-specific bug-fix code throughout the date/time files (the issue
has come up before), but once we've identified the actual cause of the
problem we can consider a workaround.
To help look at the problem do a "make clean" in that directory, and
modify the Makefile to add to CFLAGS the argument "-DDATEDEBUG" which
will enable some print statements.
If it is a glibc2 bug then presumably someone will be applying a fix to
that fairly soon. If that is the case perhaps we can either
automatically test for the deficiency in configure or we can post the
fix as a patch rather than putting it into the main source tree.
btw, can you see a problem in the regression tests? I wonder if there
are other date/time routines which show problems. I don't usually run
Postgres regression tests on my RH5.1 box since that is at work...
- Tom
RH4.2 (libc-5.3.12):
postgres=> select datetime(current_date, '11:00');
datetime
----------------------------
Fri Dec 04 11:00:00 1998 MSK
(1 row)
postgres=> select 'now'::datetime;
?column?
----------------------------
Fri Dec 04 09:18:43 1998 MSK
(1 row)
RH5.1 (glibc-2.0.7):
lockhart=> select datetime(current_date, '11:00');
datetime
----------------------------
Fri Dec 04 17:00:00 1998 MSK
(1 row)
lockhart=> select 'now'::datetime;
?column?
----------------------------
Fri Dec 04 09:16:21 1998 MSK
(1 row)
On Thu, Dec 03, 1998 at 09:59:42PM +0300, Oleg Broytmann wrote:
Well, it looked like an error in glibc2, and I continued. I wrote the
program:
...I expected to have results:
-----
MSK
MSK
MSD
1
-10800
And what did you get? I get this on Debian 2.1:
����
MEZ
CEST
1
18000
Michael
--
Dr. Michael Meskes, Manager of the Western Branch Office, Datenrevision GmbH
work: Cuxhavener Str. 36, D-21149 Hamburg, Michael.Meskes@datenrevision.de
home: Th.-Heuss-Str. 61, D-41812 Erkelenz, Michael.Meskes@usa.net
Go SF49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!
Hello!
On Fri, 4 Dec 1998, Thomas G. Lockhart wrote:
This isn't specific to your timezone; I see the same problem with
PST8PST. The routine actually getting invoked is date_datetime() in
backend/utils/adt/datetime.c. I would be reluctant to scatter
glibc2-specific bug-fix code throughout the date/time files (the issue
has come up before), but once we've identified the actual cause of the
problem we can consider a workaround.
I was pretty sure the problem is not with my timezone. The actual cause
of the problem is very simple - glibc2 has invalid timezone/tzname even
after tzset(). To my surprise, strftime() works right! It looks like
glibc2 has two different tzname's - public and internal. Because of this
ALL date/time arithmetic will fail sooner or later.
The code I stole from python and displayed yesterday works around the
problem. It seems python's configure can safely detect this and avoid. I
asked python author about this. Waiting for replay...
btw, can you see a problem in the regression tests? I wonder if there
are other date/time routines which show problems. I don't usually run
Postgres regression tests on my RH5.1 box since that is at work...
Regression test passed Ok on datetime, abstime, reltime, tinterval tests
on Debian 2.0. That is - the error is not triggered. Should we change the
regression test?
Oleg.
----
Oleg Broytmann National Research Surgery Centre http://sun.med.ru/~phd/
Programmers don't die, they just GOSUB without RETURN.
That Python code is a huge hunk of workaround kludge. I haven't looked
to see how Postgres configures itself on the glibc2 box, but we should
first check (if you haven't done so already) that it is getting the
right choices for tz configuration.
My recollection is that the two possible Unix-style setups are:
1) use global variables for time zone offsets and names. This can't work
with re-entrant or thread-safe libraries such as glibc2.
2) use variables embedded in the tz structure. I had thought that this
was an older style, but is clearly required for thread-safe code.
We should confirm that our glibc2 machines are being built with option
(2), that they don't implement a third option, and that it is indeed
massively broken. Also, we should characterize under what conditions it
is broken; so far the only case we have is when converting date to
datetime, so the problem may be within that Postgres code rather than a
general problem.
Regression test passed Ok on datetime, abstime, reltime, tinterval
tests on Debian 2.0. That is - the error is not triggered. Should we
change the regression test?
Sure we should. Send patches...
- Tom
On Fri, 4 Dec 1998, Michael Meskes wrote:
On Thu, Dec 03, 1998 at 09:59:42PM +0300, Oleg Broytmann wrote:
Well, it looked like an error in glibc2, and I continued. I wrote the
program:
...I expected to have results:
-----
MSK
MSK
MSD
1
-10800And what did you get? I get this on Debian 2.1:
О©╫О©╫О©╫О©╫
MEZ
CEST
1
18000
Definetely wrong! The first 2 lines must be identical - they are
expexted to be your timezone (MSK, in my case).
I got different results on different glibc2-based linux systems, but all
was wrong. On Debian 2.0, e.g.:
-----
MSK
EET
EST
-9200
-----
MSK is Ok here (it is output from strftime()), but other lines (global
variables) are wrong.
Thomas Lockhart gave an idea that global variables cannot be used at all
in thread-safe library, so the problem is - how to detect glibc2 at
configure stage, and how to patch date/time arithmetic in postgres. I am
working on it...
Oleg.
----
Oleg Broytmann National Research Surgery Centre http://sun.med.ru/~phd/
Programmers don't die, they just GOSUB without RETURN.
Hello!
On Fri, 4 Dec 1998, Thomas G. Lockhart wrote:
Regression test passed Ok on datetime, abstime, reltime, tinterval
tests on Debian 2.0. That is - the error is not triggered. Should we
change the regression test?Sure we should. Send patches...
I have problems creating the patch. One of my systems (Linux) returns
Thu 01 Oct 11:00:00 1998 PST
where another (Solaris) reports
Thu Oct 01 11:00:00 1998 PST
With this, datetime regression test always failed. It's minor
difference, but difference anyway.
I am always run postgres with -o -e
Oleg.
----
Oleg Broytmann http://members.tripod.com/~phd2/ phd2@earthling.net
Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann wrote:
Hello!
On Fri, 4 Dec 1998, Thomas G. Lockhart wrote:
Regression test passed Ok on datetime, abstime, reltime, tinterval
tests on Debian 2.0. That is - the error is not triggered. Should
we change the regression test?Sure we should. Send patches...
I have problems creating the patch. One of my systems (Linux) returns
Thu 01 Oct 11:00:00 1998 PST
where another (Solaris) reports
Thu Oct 01 11:00:00 1998 PST
I am always run postgres with -o -e
Uh, these differences are coming from one of your systems running on
US/NonEuropean date style and the other running with the Europena style
per your "-o -e" flags above. The Postgres date/time output routines do
not have system-specific formatting differences, other than the
differences lower down between the two main flavors of Unix libc
support.
Example follows. Talk to you soon.
- Tom
postgres=> set DateStyle = 'Postgres,NonEuropean';
SET VARIABLE
postgres=> select 'now'::datetime;
?column?
----------------------------
Tue Dec 08 04:16:28 1998 GMT
(1 row)
postgres=> set DateStyle = 'Postgres,European';
SET VARIABLE
postgres=> select 'now'::datetime;
?column?
----------------------------
Tue 08 Dec 04:16:46 1998 GMT
(1 row)
Hi!
On Fri, 4 Dec 1998, Thomas G. Lockhart wrote:
To help look at the problem do a "make clean" in that directory, and
modify the Makefile to add to CFLAGS the argument "-DDATEDEBUG" which
will enable some print statements.
Where do the output goes? To postmaster's stdout, I guess?
Oleg.
----
Oleg Broytmann National Research Surgery Centre http://sun.med.ru/~phd/
Programmers don't die, they just GOSUB without RETURN.
To help look at the problem do a "make clean" in that directory, and
modify the Makefile to add to CFLAGS the argument "-DDATEDEBUG"
which will enable some print statements.Where do the output goes? To postmaster's stdout, I guess?
Yes. I always have X up on my systems, and for debugging usually run
postmaster from the command line on one terminal while running psql from
another.
Recently, Bruce et al got me going with gdb directly on the Postgres
backend, and that works very well also. You can do the following:
$ gdb postgres
<snip>
(gdb) b datetime_out
Breakpoint 1 at 0x80ec6bc
(gdb) run -d 99
<snip>
select 'now'::datetime;
<snip>
Breakpoint 1, 0x80ec6bc in datetime_out ()
(gdb)
and you can then step through from there. Note that you should recompile
with
CFLAGS+= -O0 -g
to allow the debugger to grok the source code.
- Tom
Hello!
I think, finally, that the simplest way to solve this would be patch
configure, not datetime.c, dt.c and all that.
The struct tm on linux has tm_gmtoff and tm_zone, so it will be enough
if postgres' configure undefine HAVE_TZSET and HAVE_INT_TIMEZONE on glibc2.
Python's configure.in has the following:
# checks for structures
AC_HEADER_TIME
AC_STRUCT_TM
AC_STRUCT_TIMEZONE
and this do all neccessary defines. I think postgres' configure will
require some more tricks.
Continue investigating...
Oleg.
----
Oleg Broytmann http://members.tripod.com/~phd2/ phd2@earthling.net
Programmers don't die, they just GOSUB without RETURN.
Hi!
It looks I am doing something wrong. The file
.../src/include/port/linux.h already has it:
#if (__GLIBC__ >= 2)
#ifdef HAVE_INT_TIMEZONE
#undef HAVE_INT_TIMEZONE
#endif
but it isn't working. It is all that needs to solve the timezone
problem.
Oleg.
----
Oleg Broytmann http://members.tripod.com/~phd2/ phd2@earthling.net
Programmers don't die, they just GOSUB without RETURN.
Hi!
On Wed, 9 Dec 1998, Oleg Broytmann wrote:
#if (__GLIBC__ >= 2)
#ifdef HAVE_INT_TIMEZONE
#undef HAVE_INT_TIMEZONE
#endifbut it isn't working.
Sorry, my fault - it is working.
Another problem I have - I cannot recompile postgres with DATEDEBUG
defined in config.h - I got the error:
gcc -I../../include -I../../backend -O2 -Wall -Wmissing-prototypes -I..
-c
fmgrtab.c -o fmgrtab.o
make[2]: *** No rule to make target `adt/SUBSYS.o', needed by `SUBSYS.o'.
Stop.
make[2]: Leaving directory
`/usr/local/src/PostgreSQL/postgresql-v6.4/src/backend
/utils'
make[1]: *** [utils.dir] Error 2
make[1]: Leaving directory
`/usr/local/src/PostgreSQL/postgresql-v6.4/src/backend
'
make: *** [all] Error 2
Oleg.
----
Oleg Broytmann http://members.tripod.com/~phd2/ phd2@earthling.net
Programmers don't die, they just GOSUB without RETURN.
Another problem I have - I cannot recompile postgres with DATEDEBUG
defined in config.h - I got the error:
<snip>
These don't appear to be related. However, I usually just (slightly)
edit the src/backend/utils/adt/Makefile to add in
CFLAGS+= -DDATEDEBUG
since it is a symbol which afaik only affects files in that directory,
and should not be kept in after you are finished.
Do you have any test cases *other* than the date->datetime conversion
example? If you don't then we should probably be looking for a very
local problem, not a system-wide problem. otoh I have a hard time
picturing how this could not be a system-wide problem, given that I see
no problems on the same test case on gcc/libc5 systems.
- Tom
Hi!
On Wed, 9 Dec 1998, Thomas G. Lockhart wrote:
Another problem I have - I cannot recompile postgres with DATEDEBUG
defined in config.h - I got the error:<snip>
These don't appear to be related. However, I usually just (slightly)
I am sure this is related. When I edited config.h and commented out
DATEDEBUG the sources compiled just fine.
edit the src/backend/utils/adt/Makefile to add in
CFLAGS+= -DDATEDEBUG
Well, I'll try this.
since it is a symbol which afaik only affects files in that directory,
and should not be kept in after you are finished.
Do you have any test cases *other* than the date->datetime conversion
example? If you don't then we should probably be looking for a very
local problem, not a system-wide problem. otoh I have a hard time
picturing how this could not be a system-wide problem, given that I see
no problems on the same test case on gcc/libc5 systems.
Any use for daylight or tzname or timezone (global vars) produces
incorrect results. Again, test program is very simple:
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
int main(int argc, char **argv) {
char strf_buf[100];
time_t today_t;
struct tm * today;
memset(strf_buf, 0, 99);
today_t = time(NULL);
today = localtime(&today_t);
strftime(strf_buf, 100, "%Z", today);
printf("%s\n", strf_buf);
/* In Moscow I expect MSK/MSD/1/10800 */
printf("%s\n%s\n%d\n%ld\n", tzname[0], tzname[1], daylight, timezone);
return 0;
}
It looks like glibc2 defines these vars incorrectly. Correct values are
in struct tm (including tmzone and gmtoff).
Do you think it is local problem? I am pretty sure it is "system-wide".
Oleg.
----
Oleg Broytmann National Research Surgery Centre http://sun.med.ru/~phd/
Programmers don't die, they just GOSUB without RETURN.
I am sure this is related. When I edited config.h and commented out
DATEDEBUG the sources compiled just fine.
So send my your config.h if you want someone to look at it.
Any use for daylight or tzname or timezone (global vars) produces
incorrect results.
It looks like glibc2 defines these vars incorrectly. Correct values
are in struct tm (including tmzone and gmtoff).
Do you think it is local problem? I am pretty sure it is
"system-wide".
The glibc2 is a thread-safe library, and I would expect that the *only*
place with reliable timezone info is in the tm structure. Global
variables are not to be trusted since they are not available in a
reentrant way.
If the tm structure contains the timezone info (as it claims to on my
RH5.1 glibc2 system) then for testing try to #undef HAVE_INT_TIMEZONE in
config.h and see how it goes. If I have a chance tomorrow I'll try doing
the same at work. I'm guessing that our configure tests look for the
global variable version first, and that the glibc2 passes that test even
though the other mechanism is the right one.
- Tom
Hello!
On Thu, 10 Dec 1998, Thomas G. Lockhart wrote:
I am sure this is related. When I edited config.h and commented out
DATEDEBUG the sources compiled just fine.So send my your config.h if you want someone to look at it.
I've solved this issue. I uncomment it to
#define DATEDEBUG
but forget to add definition. After writing
#define DATEDEBUG 1
postgres compiled.
Any use for daylight or tzname or timezone (global vars) produces
incorrect results.
It looks like glibc2 defines these vars incorrectly. Correct values
are in struct tm (including tmzone and gmtoff).
Do you think it is local problem? I am pretty sure it is
"system-wide".The glibc2 is a thread-safe library, and I would expect that the *only*
place with reliable timezone info is in the tm structure. Global
variables are not to be trusted since they are not available in a
reentrant way.
Yes, I understood the idea after you explained this for a first time.
If the tm structure contains the timezone info (as it claims to on my
RH5.1 glibc2 system) then for testing try to #undef HAVE_INT_TIMEZONE in
config.h and see how it goes. If I have a chance tomorrow I'll try doing
the same at work. I'm guessing that our configure tests look for the
global variable version first, and that the glibc2 passes that test even
though the other mechanism is the right one.
#undef HAVE_INT_TIMEZONE in config.h replaced by #define HAVE_INT_TIMEZONE 1
after configure.
On the other hand, os.h (linked to port/linux.h) undefine it. I need to
test that it is still undefined when compiling datetime.c (and dt.c and all
that) - there are all provisions to use tm.tz_name/tm_gmtoff, compiler just
need right defines.
Oleg.
----
Oleg Broytmann National Research Surgery Centre http://sun.med.ru/~phd/
Programmers don't die, they just GOSUB without RETURN.
Hi!
I run postgres with DATEDEBUG and got:
GetCurrentAbsoluteTime- timezone is MSK -> -10800 seconds from UTC
date_in- input string is 1991-10-01
ParseDateTime- input string is 1991-10-01
ParseDateTime- set field[0] to 1991-10-01 type 2
DecodeDateTime- field[0] is 1991-10-01 (type 2)
DecodeNumber- 1991 is 1991 fmask=00000000 tmask=00000000
DecodeNumber- match 1991 (1991) as year
DecodeNumber- 10 is 10 fmask=00000004 tmask=00000000
DecodeNumber- match 10 (10) as month
DecodeNumber- 01 is 1 fmask=00000006 tmask=00000000
DecodeNumber- (2) match 1 (01) as day
DecodeDateTime- field[0] 1991 (00000000/0000000e) value is 1
DecodeDateTime- mask 0000000e (0000000e) set y1991 m10 d01 00:00:00
ParseDateTime- input string is 11:00
ParseDateTime- set field[0] to 11:00 type 3
DecodeTimeOnly- field[0] is 11:00 (type 3)
DecodeTimeOnly- field[0] 11:00 value is -1073763148
DecodeTimeOnly- mask 00001c0e (00001c00) 11:00:00 (0.000000)
date_datetime- date is 1991.10.01
date_datetime- time is 00:00:00 0.0000000
tm2datetime- date is -260409600.000000 (-3014.000000 0.000000 0)
tm2datetime- time is 0.000000 00:00:00 0.000000
datetime2tm- date is -260362800.000000 (2448531.000000 46800.000000)
datetime2tm- date is 1991.10.01
datetime2tm- time is 13:00:00
datetime2tm- time is 13:00:00 0.0000000
datetime2tm- (localtime) 91.09.01 15:00:00 EET dst=0
datetime2tm- date is 1991.10.01
datetime2tm- time is 15:00:00 0.0000000
EncodeDateTime- timezone is EET (EET); offset is -7200 (-10800); daylight is 0 (0
EncodeDateTime- day is 2448531
EncodeDateTime- date result is Tue 01 Oct 15:00:00 1991 EET
The last 3 lines shows an error. Continue searching...
Oleg.
----
Oleg Broytmann National Research Surgery Centre http://sun.med.ru/~phd/
Programmers don't die, they just GOSUB without RETURN.
"Thomas G. Lockhart" <lockhart@alumni.caltech.edu> writes:
If the tm structure contains the timezone info (as it claims to on my
RH5.1 glibc2 system) then for testing try to #undef HAVE_INT_TIMEZONE in
config.h and see how it goes. If I have a chance tomorrow I'll try doing
the same at work. I'm guessing that our configure tests look for the
global variable version first, and that the glibc2 passes that test even
though the other mechanism is the right one.
Yes, the autoconf test just checks for a global variable "timezone"
declared in <time.h>. It'd probably be better to reverse the test to
check for the timezone fields in a struct tm. I don't much care for
the name of the configuration symbol either ... maybe change it to
HAVE_STRUCT_TM_TZ_FIELDS ?
regards, tom lane
Import Notes
Reply to msg id not found: YourmessageofThu10Dec1998054553+0000366F6011.AAA68CD5@alumni.caltech.edu | Resolved by subject fallback
Hi!
On Thu, 10 Dec 1998, Tom Lane wrote:
Yes, the autoconf test just checks for a global variable "timezone"
declared in <time.h>. It'd probably be better to reverse the test to
check for the timezone fields in a struct tm. I don't much care for
the name of the configuration symbol either ... maybe change it to
HAVE_STRUCT_TM_TZ_FIELDS ?
It is enought to add AC_STRUCT_TIMEZONE to configure.in - and after it
configure will #define HAVE_TM_ZONE 1 in config.h.
I already did it - just haven't submitted the patch.
Oleg.
----
Oleg Broytmann http://members.tripod.com/~phd2/ phd2@earthling.net
Programmers don't die, they just GOSUB without RETURN.