Redhat 7.3 time manipulation bug

On Saturday 18 May 2002 12:34 am, Manuel Sugawara wrote:

Something is pretty broken in redhat 7.3 but I'm not sure what and I
don't have time to dig further

Both cases works correctly in Redhat 7.2. Sorry if this is not the
correct forum however an alert is nice for people planning an Redhat
upgrade. I hope to see pretty soon an update since I don't want to
downgrade my server.

Filed on Red Hat's Bugzilla system as bug# 65227, as I can reliably reporoduce
this bug here, and PostgreSQL 7.2.1 on Red Hat 6.2 on SPARC does not exhibit
the bug.

Tom (or Thomas):

Where would we go to ferret out the source of this bug? More to the point: we
need a test case in C that could expose this as a glibc bug. Methinks Red
Hat would want this bug ferretted out, as it would likely cause problems with
RedHat Database on RH 7.3's glibc.....
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen <lamar.owen@wgcr.org> writes:

Filed on Red Hat's Bugzilla system as bug# 65227, as I can reliably
reporoduce this bug here, and PostgreSQL 7.2.1 on Red Hat 6.2 on
SPARC does not exhibit the bug.

Thanks for filling that report. I couldn't remember what had forgotten
;-)

Regards,
Manuel.

Lamar Owen <lamar.owen@wgcr.org> writes:

Where would we go to ferret out the source of this bug? More to the
point: we need a test case in C that could expose this as a glibc
bug.

Seems like mktime(3) is having problems with dates before the
epoch. Attached is the a program to test this. The glibc source is now
downloading I will try to hunt down this bug but not until the next
week.

Regards,
Manuel.

On Monday 20 May 2002 08:08 pm, Manuel Sugawara wrote:

Where would we go to ferret out the source of this bug? More to the
point: we need a test case in C that could expose this as a glibc
bug.

Seems like mktime(3) is having problems with dates before the
epoch. Attached is the a program to test this. The glibc source is now
downloading I will try to hunt down this bug but not until the next
week.

It's not a bug. At least not according to the ISO C standard. See
http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap04.html#tag_04_14
for the definition of 'Seconds Since the Epoch', then cross-reference to the
man page of mktime.

I don't like it any more than you do, but that is the letter of the standard.

Thomas, any comments?

Our implementation is broken, then. Thomas, is this fixable for a 7.2.x
release, or something for 7.3?
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

...

Our implementation is broken, then. Thomas, is this fixable for a 7.2.x
release, or something for 7.3?

"Our implementation is broken, then" is really not a conclusion to be
reached. The de facto behavior of mktime() on all world-class
implementations is to support pre-1970 times. This has been true forever
afaik, certainly far longer than PostgreSQL (or Postgres) has been in
existance.

Any standard which chooses to ignore pre-1970 dates is fundamentally
broken imho, and I'm really ticked off that the glibc folks have so
glibly introduced a change of this nature and magnitude without further
discussion.

- Thomas

Lamar Owen <lamar.owen@wgcr.org> writes:

On Monday 20 May 2002 08:08 pm, Manuel Sugawara wrote:

Where would we go to ferret out the source of this bug? More to the
point: we need a test case in C that could expose this as a glibc
bug.

Seems like mktime(3) is having problems with dates before the
epoch. Attached is the a program to test this. The glibc source is now
downloading I will try to hunt down this bug but not until the next
week.

It's not a bug. At least not according to the ISO C standard. See
http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap04.html#tag_04_14
for the definition of 'Seconds Since the Epoch', then
cross-reference to the man page of mktime.

I see. This behavior is consistent with the fact that mktime is
supposed to return -1 on error, but then is broken in every other Unix
implementation that I know.

Any other workaround than downgrade or install FreeBSD?

Regards,
Manuel.

On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote:

I see. This behavior is consistent with the fact that mktime is
supposed to return -1 on error, but then is broken in every other Unix
implementation that I know.

Any other workaround than downgrade or install FreeBSD?

Complain to Red Hat. Loudly. However, as this is a glibc change, other
distributors are very likely to fold in this change sooner rather than later.

Try using timestamp without timezone?
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

On Tue, 21 May 2002, Lamar Owen wrote:

On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote:

I see. This behavior is consistent with the fact that mktime is
supposed to return -1 on error, but then is broken in every other Unix
implementation that I know.

Any other workaround than downgrade or install FreeBSD?

Complain to Red Hat. Loudly. However, as this is a glibc change, other
distributors are very likely to fold in this change sooner rather than
later.

Relying on nonstandardized/nondocumented behaviour is a program bug, not a
glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
it, but glibc is not the component with a problem.

--
Trond Eivind Glomsr�d
Red Hat, Inc.

On Tuesday 21 May 2002 12:31 pm, Trond Eivind Glomsr�d wrote:

On Tue, 21 May 2002, Lamar Owen wrote:

However, as this is a glibc change, other
distributors are very likely to fold in this change sooner rather than
later.

Relying on nonstandardized/nondocumented behaviour is a program bug, not a
glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
it, but glibc is not the component with a problem.

In your opinion. Not everyone agrees with the new glibc behavior. In fact,
some here are rather livid about it. But that's a digression. The matter at
hand is making it work right again.

It seems to me someone should have thought about this before making the glibc
change that had the potential for breaking a large application -- regardless
of who is at fault, it's egg in Red Hat's face for not catching it sooner
(and egg in my face as well, as I must admit that I of all people should have
caught this earlier).

Is the change in glibc behavior noted in the release notes? The man page
isn't changed either, from Red Hat 6.2, in fact. The only change is adhering
to the ISO definition of 'Seconds Since the Epoch' rather than the defacto
industry-accepted definition that has been with us a very long time.

Like PostgreSQL's refusal to upgrade in a sane manner, this should have
received release notes billing, IMHO. Then I, nor anyone else, could
reasonably complain.

But this does show the need for the regression testing packge, no? :-) And it
also shows the danger in becoming too familiar with certain regression tests
failing due to locale -- to the extent that a locale issue was the first
thing thought of. To the extent that I'm going to change my build process to
include regression testing as a part of the build -- and any failure will
abort the build. Maybe that will get my attention. And anyone else's
rebuilding from the source RPM. SuSE already does this. I wonder how
they've handled this issue with 8.0?

In any case, this isn't just a Red Hat problem, as it's going to cause
problems with the use of timestamps on ANY glibc 2.2.5 dist. That's more
than Red Hat, by a large margin.

And I think that it is naive to think that PostgreSQL is the only program that
has used mktime's behavior in the negative-time_t zone. Other packages are
likely impacted, to a greater or lesser extent.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

On Tue, 2002-05-21 at 21:31, Trond Eivind Glomsrød wrote:

On Tue, 21 May 2002, Lamar Owen wrote:

On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote:

I see. This behavior is consistent with the fact that mktime is
supposed to return -1 on error, but then is broken in every other Unix
implementation that I know.

Any other workaround than downgrade or install FreeBSD?

Complain to Red Hat. Loudly. However, as this is a glibc change, other
distributors are very likely to fold in this change sooner rather than
later.

Relying on nonstandardized/nondocumented behaviour is a program bug, not a
glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
it, but glibc is not the component with a problem.

Still it seems kind of silly to have a function that works different
from all other implementations and forces people to use their own
function of the same name (lifted from BSD and also compliant).

Speaking of nonstandardized/nondocumented behaviour, I read from "The
Open Sources" book that if you implement TCP/IP strictly from the RFCs
then it won't interoperate with any other TCP/IP stack.

I hope that Red Hat is not going to be "standards compliant" here ;)

--------------
Hannu

Trond Eivind Glomsrød <teg@redhat.com> writes:

Relying on nonstandardized/nondocumented behaviour is a program bug,
not a glibc bug.

The question is: how this thing didn't show up before? ISTM that
someone is not doing his work correctly.

PostgreSQL needs fixing.

Arguably, however, right now is *a lot easier* to fix glibc, and it's
really needed for production systems using postgreSQL and working on
RedHat. But redhat users doesn't matter, the most important thing is
*strict* conformace to standars, right?

Since we ship both, we're looking at it, but glibc is not the

^^^^^^^^^^^^^^^^^^^
The sad true is: you only answered when the 'Complain to Red Hat'
statement appeared, not a single word before and not a single word
when the bug report were closed. I'm really disappointed.

The nice thing is: glibc is free software and we don't have to wait or
relay on some of the redhat staff members (thanks god) for this to get
fixed or say: for the standard to get extended again. The patch to
glibc is pretty straightforward and attached.

Regards,
Manuel.

On 21 May 2002, Manuel Sugawara wrote:

Trond Eivind Glomsr�d <teg@redhat.com> writes:

Relying on nonstandardized/nondocumented behaviour is a program bug,
not a glibc bug.

The question is: how this thing didn't show up before? ISTM that
someone is not doing his work correctly.

FWIW, I ran the regressions tests some time ago(probably before that
change to glibc) . Since the tests are known
to be broken wrt. time issues anyway (as well as currency, math and sorting),
it's easy to overlook.

PostgreSQL needs fixing.

Arguably, however, right now is *a lot easier* to fix glibc, and it's
really needed for production systems using postgreSQL and working on
RedHat.

You're not "fixing" glibc, you're reintroducing non-standardized, upstream
removed behaviour. That's typically a very bad thing. If anything, it
demonstrates the importance of not using or relying on
unstandardized/undocumented behaviour (and given that time_t is pretty
restrictive anyway, you'll need something else to keep dates. It doesn't
even cover all living people, and definitely not historical dates).

Since we ship both, we're looking at it, but glibc is not the

^^^^^^^^^^^^^^^^^^^
The sad true is: you only answered when the 'Complain to Red Hat'
statement appeared, not a single word before and not a single word
when the bug report were closed. I'm really disappointed.

The bug wasn't open for long, and was closed by someone else.

The nice thing is: glibc is free software

Also, notice that this was where the fix came from: The upstream
maintainers (some of whom work for us, others don't).

--
Trond Eivind Glomsr�d
Red Hat, Inc.

On Tuesday 21 May 2002 03:09 pm, Trond Eivind Glomsr�d wrote:

FWIW, I ran the regressions tests some time ago(probably before that
change to glibc) . Since the tests are known
to be broken wrt. time issues anyway (as well as currency, math and
sorting), it's easy to overlook.

The time tests have never broken in this manner before on Red Hat. When the
original regression failure report was posted, I saw right away that this was
not the run of the mill locale issue -- this was a real problem. Regression
testing must become a regularly scheduled activity, methinks. In the RPM
build process, we can control the locale to the extent that the tests will
pass (except on DST days) reliably. I am going to implement this for my next
RPM set. Along with a patch to this problem -- we _can_ patch around this, I
believe, but it's not likely going to be an easy one.

We have gotten blind to the regular locale-induced failures -- this is not a
good thing.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

=?ISO-8859-1?Q?Trond_Eivind_Glomsr=F8d?= <teg@redhat.com> writes:

Relying on nonstandardized/nondocumented behaviour is a program bug, not a
glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
it, but glibc is not the component with a problem.

A library that can no longer cope with dates before 1970 is NOT my idea
of a component without a problem. We will be looking at ways to get
around glibc's breakage at the application level, since we have little
alternative other than to declare Linux an unsupported platform;
but it's still glibc (and the ISO spec:-() that are broken.

regards, tom lane

On Tue, 2002-05-21 at 18:24, Lamar Owen wrote:

In any case, this isn't just a Red Hat problem, as it's going to cause
problems with the use of timestamps on ANY glibc 2.2.5 dist. That's more
than Red Hat, by a large margin.

I'm running glibc 2.2.5 on Debian and all regression tests pass OK (with
make check). I don't see any note in the glibc Debian changelog about
reversing an upstream change to mktime().

I missed the first messages in this thread and I can't find them in the
archive. What should I be looking for to see if I have the problem you
have encountered or to see why I don't have it if I ought to have?

--
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C

"O come, let us worship and bow down; let us kneel
before the LORD our maker." Psalms 95:6

On Tuesday 21 May 2002 06:09 pm, Oliver Elphick wrote:

On Tue, 2002-05-21 at 18:24, Lamar Owen wrote:

In any case, this isn't just a Red Hat problem, as it's going to cause
problems with the use of timestamps on ANY glibc 2.2.5 dist. That's more
than Red Hat, by a large margin.

I'm running glibc 2.2.5 on Debian and all regression tests pass OK (with
make check). I don't see any note in the glibc Debian changelog about
reversing an upstream change to mktime().

I missed the first messages in this thread and I can't find them in the
archive. What should I be looking for to see if I have the problem you
have encountered or to see why I don't have it if I ought to have?

Hmmm. Compile and run the attached program. If you get -1, it's the new
behavior. It might be interesting to see the differences here.....
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Manuel Sugawara <masm@fciencias.unam.mx> writes:

+#if 0
/* Only years after 1970 are defined.
If year is 69, it might still be representable due to
timezone differnces. */
if (year < 69)
return -1;
+#endif

Hm. If that fixes it, it implies that all the other support for
pre-1970 dates is still there (notably, entries in the timezone tables).

Should we assume that future glibc releases will rip out the timezone
database entries and other support for pre-1970 dates? Or is the
breakage going to stop with mktime?

regards, tom lane

Lamar Owen writes:

SuSE already does this. I wonder how they've handled this issue with
8.0?

Their glibc doesn't have that problem.

Personally, I think if you need time (zone) support before 1970, obtain
one of the various operating systems that support it. There's little
value in hacking around it in PostgreSQL, since the rest of your system
will be broken as well.

--
Peter Eisentraut peter_e@gmx.net

SuSE already does this. I wonder how they've handled this issue with
8.0?

Their glibc doesn't have that problem.

My strong recollection is that a SuSE guy was the one applying the
change. So this is coming to those systems too. I may not remember that
correctly though...

Personally, I think if you need time (zone) support before 1970, obtain
one of the various operating systems that support it. There's little
value in hacking around it in PostgreSQL, since the rest of your system
will be broken as well.

Yes, I'm afraid I agree. In practice, maybe most applications won't
notice. But after getting the Linux time zone databases set up to be
better than most (Solaris has the best I've found for fidelity to
pre-1970 year-to-year conventions) throwing that work away is just plain
silly. I consider this a major gaff on the part of the commercial Linux
houses to not see this coming and to contribute to a better solution.

- Thomas

On Tue, 2002-05-21 at 23:47, Lamar Owen wrote:

Hmmm. Compile and run the attached program. If you get -1, it's the new
behavior. It might be interesting to see the differences here.....

$ a.out
The system thinks 11/30/1969 is a timestamp of -176400
$ dpkg -l libc6
...
||/ Name Version Description
+++-==============-==============-============================================
ii libc6 2.2.5-6 GNU C Library: Shared libraries and Timezone

--
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C

"We are troubled on every side, yet not distressed; we
are perplexed, but not in despair; persecuted, but not
forsaken; cast down, but not destroyed; Always bearing
about in the body the dying of the Lord Jesus, that
the life also of Jesus might be made manifest in our
body." II Corinthians 4:8-10

On Wed, 2002-05-22 at 02:14, Tom Lane wrote:

=?ISO-8859-1?Q?Trond_Eivind_Glomsr=F8d?= <teg@redhat.com> writes:

Relying on nonstandardized/nondocumented behaviour is a program bug, not a
glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
it, but glibc is not the component with a problem.

A library that can no longer cope with dates before 1970 is NOT my idea
of a component without a problem. We will be looking at ways to get
around glibc's breakage at the application level, since we have little
alternative other than to declare Linux an unsupported platform;
but it's still glibc (and the ISO spec:-() that are broken.

IIRC the spec is not _really_ broken - it still allows the correct
behaviour :)

The fact the ISO spec is broken usually means that at least one of the
big vendors involved in ISO spec creation must have had a broken
implementation at that time.

Most likely they have fixed it by now ...

Does anyone know _any_ other libc that has this behaviour ?

--------------
Hannu

IIRC the spec is not _really_ broken - it still allows the correct
behaviour :)

Yes.

The fact the ISO spec is broken usually means that at least one of the
big vendors involved in ISO spec creation must have had a broken
implementation at that time.

Right. IBM.

Most likely they have fixed it by now ...

Nope, though I don't know for sure. Anyone here have a recent AIX
machine to test?

Does anyone know _any_ other libc that has this behaviour ?

AIX and (I think) Irix.

Trond, do you have a suggestion on how to get this addressed at the
glibc level? Does someone within RH participate in glibc development? If
so, can we get them to champion changes which would comply with the
standard but remove this arbitrary breakage?

The changes would involve returning -1 from mktime() for dates before
1970, and using the tm_isdst flag to indicate whether a time zone
translation was not possible.

- Thomas

Thomas Lockhart writes:

Right. IBM.

Most likely they have fixed it by now ...

Nope, though I don't know for sure. Anyone here have a recent AIX
machine to test?

Well on AIX 4.3.3 the output from Lamar's earlier test program is:

The system thinks 11/30/1969 is a timestamp of -1

and tm_isdst is left at -1...

I could boot the machine into 5.0 too, but going by the AIX 5L
manpages it still returns -1:

Note: The mktime subroutine cannot convert time values before
00:00:00 UTC, January 1, 1970 and after 03:14:07 UTC, January 19,
2038.

And getting an Irix 5.3 box up and running would be a chore!

Lee.

...

AIX and (I think) Irix.

How do we currently support AIX/Irix ?

Dates and times prior to 1970 have no time zone (they are in GMT, as is
the case for all platforms on dates before 1903 and after 2038). We have
separate regression test results for those platforms.

- Tom

On Wed, 22 May 2002, Thomas Lockhart wrote:

IIRC the spec is not _really_ broken - it still allows the correct
behaviour :)

Yes.

The fact the ISO spec is broken usually means that at least one of the
big vendors involved in ISO spec creation must have had a broken
implementation at that time.

Right. IBM.

Most likely they have fixed it by now ...

Nope, though I don't know for sure. Anyone here have a recent AIX
machine to test?

Does anyone know _any_ other libc that has this behaviour ?

AIX and (I think) Irix.

Trond, do you have a suggestion on how to get this addressed at the
glibc level? Does someone within RH participate in glibc development?

Jakub Jelinek, Ulrich Drepper and others.

If so, can we get them to champion changes which would comply with the
standard but remove this arbitrary breakage?

Unlikely. They already saw (and participated, at least Ulrich) a thread on
this with Lamar. Their take is "this is the standard, you should do what
the standard says and not rely on
undocumented, non-standardized sideeffects.

The changes would involve returning -1 from mktime() for dates before
1970, and using the tm_isdst flag to indicate whether a time zone
translation was not possible.

--
Trond Eivind Glomsr�d
Red Hat, Inc.

On Wed, 2002-05-22 at 15:30, Thomas Lockhart wrote:

IIRC the spec is not _really_ broken - it still allows the correct
behaviour :)

Yes.

The fact the ISO spec is broken usually means that at least one of the
big vendors involved in ISO spec creation must have had a broken
implementation at that time.

Right. IBM.

Most likely they have fixed it by now ...

Nope, though I don't know for sure. Anyone here have a recent AIX
machine to test?

Does anyone know _any_ other libc that has this behaviour ?

AIX and (I think) Irix.

How do we currently support AIX/Irix ?

----------------
Hannu

On Wed, 2002-05-22 at 08:00, Thomas Lockhart wrote:

...

If so, can we get them to champion changes which would comply with the
standard but remove this arbitrary breakage?

Unlikely. They already saw (and participated, at least Ulrich) a thread on
this with Lamar. Their take is "this is the standard, you should do what
the standard says and not rely on undocumented, non-standardized sideeffects.

OK. They must be new guys.

:-) Very funny.

--
---------------. ,-. 1325 Chesapeake Terrace
Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA
Red Hat `--' drepper at redhat.com `------------------------

On Wednesday 22 May 2002 01:12 pm, Ulrich Drepper wrote:

On Wed, 2002-05-22 at 08:00, Thomas Lockhart wrote:

If so, can we get them to champion changes which would comply with
the standard but remove this arbitrary breakage?

Unlikely. They already saw (and participated, at least Ulrich) a thread
on this with Lamar. Their take is "this is the standard, you should do
what the standard says and not rely on undocumented, non-standardized
sideeffects.

OK. They must be new guys.

:-) Very funny.

What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5
(same as Red Hat 7.3's version). They are different. And, IMO, those
results are the 'expected' results on a unixoid system, ISO or no ISO.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

On Wed, 2002-05-22 at 10:51, Lamar Owen wrote:

What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5
(same as Red Hat 7.3's version).

This is a completely different version. Once Debian updates (in a few
years) they'll get the same result.

If you are misusing interfaces you get what you deserve. At no time was
it correct to use these functions for general date manipulation. It
always only was allowed to use them to represent system times and there
was no Unix system before the epoch. Therefore you argumentation is
completely wrong.

If you need date manipulation write your own code which work for all the
times you want to represent.

--
---------------. ,-. 1325 Chesapeake Terrace
Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA
Red Hat `--' drepper at redhat.com `------------------------

Ulrich Drepper <drepper@redhat.com> writes:

If you are misusing interfaces you get what you deserve. At no time was
it correct to use these functions for general date manipulation. It
always only was allowed to use them to represent system times and there
was no Unix system before the epoch. Therefore you argumentation is
completely wrong.

If you need date manipulation write your own code which work for all the
times you want to represent.

We may indeed end up doing that, if glibc fails to provide the
functionality we need; but your argument is nonsense. Unix systems have
*always* interpreted time_t as a signed offset from the epoch. Do you
really think that when Unixen were first built in the early 70s, there
was no interest in working with pre-1970 dates? Hardly likely.

glibc has just taken a large step backwards by comparison to every other
libc on the planet. Claiming that you are okay because you conform to
a lowest-common-denominator ISO spec is beside the point. ISO specs are
only pieces of paper. You have removed functionality that is de facto
standard, important in practice, and *provided by most of your
competition*. People will start going to the competition. Or perhaps
there will be a glibc fork.

Postgres is not the only application that will be complaining loudly
about this change; anyone who has to deal with historical information
is going to be unhappy. We just happen to be the earliest bearers of
the bad news. But you will end up reverting this change due to pushback
from users. Want to make a side bet?

regards, tom lane

On Wed, 2002-05-22 at 11:23, Tom Lane wrote:

Unix systems have
*always* interpreted time_t as a signed offset from the epoch.

No. This always was an accident if it happens.

Do you
really think that when Unixen were first built in the early 70s, there
was no interest in working with pre-1970 dates? Hardly likely.

There never were files or any system events with these dates. Yes.

And just to educate you and your likes: the majority of systems on this
planet use mktime this way. I hate using this as an argument, but
beside major Unixes M$ systems also do this.

But you will end up reverting this change due to pushback
from users. Want to make a side bet?

Sure. Especially not everybody is that stubborn.

--
---------------. ,-. 1325 Chesapeake Terrace
Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA
Red Hat `--' drepper at redhat.com `------------------------

On Wednesday 22 May 2002 01:58 pm, Ulrich Drepper wrote:

On Wed, 2002-05-22 at 10:51, Lamar Owen wrote:

What isn't funny is Oliver Elphick's results on Debian, running glibc
2.2.5 (same as Red Hat 7.3's version).

This is a completely different version. Once Debian updates (in a few
years) they'll get the same result.

A completely different version with the same version number? Or is this a
case of a Red Hat version number really meaning something different
Shouldn't glibc 2.2.5 be the same as glibc 2.2.5 regardless of distribution?

And who's to stop them patching out the new stuff and reinstating the old
behavior? :-)

If you are misusing interfaces you get what you deserve. At no time was
it correct to use these functions for general date manipulation. It
always only was allowed to use them to represent system times and there
was no Unix system before the epoch. Therefore you argumentation is
completely wrong.

If it is completely wrong, then tell Sun, HP, and all the rest of the Unix
vendors, including the authors of the original AT&T code as lifted by
Berkeley, that they're wrong and you're right. They'll laugh you to scorn.

And just which 'major Unixen' other than AIX and Irix that follow the letter
of the 'seconds since the epoch' definition of the ISO standard?

If you need date manipulation write your own code which work for all the
times you want to represent.

The mktime bug doesn't effect our representation of dates and times other than
the timezone at this time. What's aggravating to me is the surprise factor.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

This thread is getting pretty annoying rather than constructive. By
the mean time I can see the users of many db's running under linux
loudly complaining.

As a user of both products (glibc and postgres), I would like to see a
good compromise in both sides. For instance: postgreSQL will implement
it's own time engine in one or to releases (In a few months I can do
it if no one else volunteers) and glibc will revert this change for
one or two releases (I can do it right now). That way everyone will be
happy; particularly me and many others using glibc and postgres.

Regards,
Manuel.

OK. They must be new guys.

:-) Very funny.

Hey, it worked. Got you out of the woodwork. :))

- Thomas

On Wed, 2002-05-22 at 15:30, Thomas Lockhart wrote:

IIRC the spec is not _really_ broken - it still allows the correct
behaviour :)

Yes.

The fact the ISO spec is broken usually means that at least one of the
big vendors involved in ISO spec creation must have had a broken
implementation at that time.

Right. IBM.

Most likely they have fixed it by now ...

Nope, though I don't know for sure. Anyone here have a recent AIX
machine to test?

Does anyone know _any_ other libc that has this behaviour ?

AIX and (I think) Irix.

How do we currently support AIX/Irix ?

Why should we rely on broken glibc and the standard? Why don't we make
our own mktime() and use it on all platforms.
--
Tatsuo Ishii

...

Why should we rely on broken glibc and the standard? Why don't we make
our own mktime() and use it on all platforms.

The downside to doing that is that we then take over maintenance of the
code and, more importantly, the timezone database.

But it might be the best thing to do. It looks like the zic package
pointed to the other day could be used, though we may have to change
some of the variables and entry points to avoid library conflicts. But
we could also blow past the y2038 limit afaict which would be nice.

- Thomas

Thomas Lockhart <lockhart@fourpalms.org> writes:

Why should we rely on broken glibc and the standard? Why don't we make
our own mktime() and use it on all platforms.

The downside to doing that is that we then take over maintenance of the
code and, more importantly, the timezone database.

But it might be the best thing to do.

I've been sorta thinking the same thing. We could get out from under
the Y2038 issue, and also eliminate a whole lot of platform
dependencies. Not to mention sillinesses like being unable to recognize
a bad timezone name when it's fed to us.

Exactly how much work (and code bulk) would we be taking on? I've
never looked at how big the timezone databases are...

regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> wrote:
[snip]

Exactly how much work (and code bulk) would we be taking on? I've
never looked at how big the timezone databases are...

Some answers on database sizes, if this is any help...
I did "du -sh /usr/share/zoneinfo/" on them all.

OpenBSD 3.1, sparc64:
1.3M /usr/share/zoneinfo/

Linux, i686, oldish mandrake (6.x?), glibc 2.1.3:
478k /usr/share/zoneinfo

Linux, i686, newish redhat 7.2, glibc 2.2.4:
4.9M /usr/share/zoneinfo

Linux, alpha EV56, oldish redhat 6.2, glibc 2.1.3
1.4M /usr/share/zoneinfo

regards, tom lane

Magnus

--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Programmer/Networker [|] Magnus Naeslund
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

On Thu, May 23, 2002 at 04:42:13AM +0200, Magnus Naeslund(f) wrote:

Some answers on database sizes, if this is any help...
I did "du -sh /usr/share/zoneinfo/" on them all.

OpenBSD 3.1, sparc64:
1.3M /usr/share/zoneinfo/

Linux, i686, oldish mandrake (6.x?), glibc 2.1.3:
478k /usr/share/zoneinfo

Linux, i686, newish redhat 7.2, glibc 2.2.4:
4.9M /usr/share/zoneinfo

^^^^

What do they do with that di?

Linux, alpha EV56, oldish redhat 6.2, glibc 2.1.3
1.4M /usr/share/zoneinfo

Here's what my Debian Woody system says:

1.5M /usr/share/zoneinfo

And this is with glibc 2.2.5. Of course this wouldn't be the first time
that RedHat uses the same version number for a different version.

Michael
--
Michael Meskes
Michael@Fam-Meskes.De
Go SF 49ers! Go Rhein Fire!
Use Debian GNU/Linux! Use PostgreSQL!

On Wed, May 22, 2002 at 10:58:15AM -0700, Ulrich Drepper wrote:

On Wed, 2002-05-22 at 10:51, Lamar Owen wrote:

What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5
(same as Red Hat 7.3's version).

This is a completely different version. Once Debian updates (in a few
years) they'll get the same result.

Ulrich, how shall I understand this? I'm pretty sure Oliver
does not use a Debian 2.2 system with glibc-2.1.3 but a pretty
up-to-date one. The glibc version in the soon to be released Woody
release is 2.2.5.

This seems to be the very same version that RedHat uses. So what
could/should Debian update? Besides, the "in a few years" comment looks
like FUD to me. It may be a few years since we talked the last time, but
I cannot imagine you changed that much that you spread FUD
nowadays. So I probably misunderstood this sentence, but nevertheless
would like to know what Debian should update.

Or do you mean that once Debian updates to glibc 2.3 (or whatever the
next release will be) it will show the same results? Does RedHat 7.3
already run on that new release? But then I would think they changed the
version number.

Michael

--
Michael Meskes
Michael@Fam-Meskes.De
Go SF 49ers! Go Rhein Fire!
Use Debian GNU/Linux! Use PostgreSQL!

On 22 May 2002, Ulrich Drepper wrote:

On Wed, 2002-05-22 at 10:51, Lamar Owen wrote:

What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5
(same as Red Hat 7.3's version).

This is a completely different version. Once Debian updates (in a few
years) they'll get the same result.

If you are misusing interfaces you get what you deserve. At no time was
it correct to use these functions for general date manipulation. It
always only was allowed to use them to represent system times and there
was no Unix system before the epoch. Therefore you argumentation is
completely wrong.

If you need date manipulation write your own code which work for all the
times you want to represent.

We are Redhat, you will be assimilated

On 22 May 2002, Ulrich Drepper wrote:

On Wed, 2002-05-22 at 11:23, Tom Lane wrote:

Unix systems have
*always* interpreted time_t as a signed offset from the epoch.

No. This always was an accident if it happens.

Do you
really think that when Unixen were first built in the early 70s, there
was no interest in working with pre-1970 dates? Hardly likely.

There never were files or any system events with these dates. Yes.

And just to educate you and your likes: the majority of systems on this
planet use mktime this way. I hate using this as an argument, but
beside major Unixes M$ systems also do this.

M$ systems crashes regularly too ... is Redhat going to adopt that too?

--=-Z1lifK4QZqKV8kHqHfYT
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Wed, 2002-05-22 at 10:51, Lamar Owen wrote:

What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2=

.5=20

(same as Red Hat 7.3's version).

This is a completely different version. Once Debian updates (in a few
years) they'll get the same result.

If you are misusing interfaces you get what you deserve. At no time was
it correct to use these functions for general date manipulation. It
always only was allowed to use them to represent system times and there
was no Unix system before the epoch. Therefore you argumentation is
completely wrong.

If you need date manipulation write your own code which work for all the
times you want to represent.

This is indeed a problem with dates on LIBC, because even if it is
theoretically satisfactory to describe dates within some range within 2^31
seconds of 1970, that is certainly NOT satisfactory for describing all dates
of interest unless you're being terribly parochial about what is to be
considered "of interest."

My father's birth falls within 2^31 seconds of 1970, but the Battle of
Agincourt doesn't.

Any backup of any Unix system in history falls within 2^31 seconds of 1970,
but there are people alive whose births don't.

People get away with using Unix dates as a "general" date type when they don't
have to work outside a limited range. If/when there is a move to 64 bit
values, that will provide something with enough range to cover history back to
ridiculously early times, relieving the limit.

But anybody using Unix dates as "general dates" has leaped into exactly the
same sort of trap that caused people to get so paranoid about Y2K.
--
(concatenate 'string "cbbrowne" "@acm.org")
http://www.cbbrowne.com/info/oses.html
Do Roman paramedics refer to IV's as "4's"?

--
(concatenate 'string "aa454" "@freenet.carleton.ca")
http://www.ntlug.org/~cbbrowne/linuxxian.html
"So, when you typed in the date, it exploded into a sheet of blue
flame and burned the entire admin wing to the ground? Yes, that's a
known bug. We'll be fixing it in the next release. Until then, try not
to use European date format, and keep an extinguisher handy."
-- slam@pobox.com (Tequila Rapide)

On 22 May 2002, Ulrich Drepper wrote:

On Wed, 2002-05-22 at 11:23, Tom Lane wrote:

Unix systems have
*always* interpreted time_t as a signed offset from the epoch.

No. This always was an accident if it happens.

Do you
really think that when Unixen were first built in the early 70s, there
was no interest in working with pre-1970 dates? Hardly likely.

There never were files or any system events with these dates. Yes.

And just to educate you and your likes: the majority of systems on this
planet use mktime this way. I hate using this as an argument, but
beside major Unixes M$ systems also do this.

M$ systems crashes regularly too ... is Redhat going to adopt that too?

Harbison and Steele indicates that:

"Although the traditional return type of time is long, the value returned is
usually of type unsigned long."

That is NOT a "Linux" reference; it was published before Linus had started
working on his kernel project.

ANSI does not indicate that time_t is a signed int, signed long, or whatever.
It is only defined to be an arithmetic type.

It would not be a bug for GLIBC to define time_t to be an unsigned int, with
an epoch beginning of January 1, 1999. That definition would conform to ANSI
C.

Since that definition can conform to ANSI C, and Unix systems have more
particularly been known to use unsigned ints with epoch of 1970, it is NOT
reasonable to assume that time_t can be used to express dates that are at all
ancient in the past.

Indeed, it is fairly _useful_ for different libc implementations to make
different assumptions about things whose definitions are permitted to vary, as
this makes it easier to call out as WRONG those systems that make up their own
definitions.

People will no doubt get defensive about their own non-standard
implementations of things; it is certainly far easier to cry "They're trying
to play Microsoft!" than it is to be honest and actually look at the standards.
--
(concatenate 'string "cbbrowne" "@ntlug.org")
http://www.cbbrowne.com/info/advocacy.html
When confronted by a difficult problem, solve it by reducing it to the
question, "How would the Lone Ranger handle this?"

--
(reverse (concatenate 'string "gro.mca@" "enworbbc"))
http://www.cbbrowne.com/info/linuxxian.html
As of next Monday, COMSAT will be flushed in favor of a string and two tin
cans. Please update your software.

On Thu, 2002-05-23 at 07:20, Michael Meskes wrote:

The glibc version in the soon to be released Woody
release is 2.2.5.

The version in RHL7.3 is 2.2.5-34. This is not what Debian uses. Maybe
you should read the changelog for the version.

--
---------------. ,-. 1325 Chesapeake Terrace
Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA
Red Hat `--' drepper at redhat.com `------------------------

On Thu, 23 May 2002 cbbrowne@cbbrowne.com wrote:

On 22 May 2002, Ulrich Drepper wrote:

On Wed, 2002-05-22 at 11:23, Tom Lane wrote:

Unix systems have
*always* interpreted time_t as a signed offset from the epoch.

No. This always was an accident if it happens.

Do you
really think that when Unixen were first built in the early 70s, there
was no interest in working with pre-1970 dates? Hardly likely.

There never were files or any system events with these dates. Yes.

And just to educate you and your likes: the majority of systems on this
planet use mktime this way. I hate using this as an argument, but
beside major Unixes M$ systems also do this.

M$ systems crashes regularly too ... is Redhat going to adopt that too?

< stuff deleted >

People will no doubt get defensive about their own non-standard
implementations of things; it is certainly far easier to cry "They're trying
to play Microsoft!" than it is to be honest and actually look at the standards.

Just to clarify, if this was directed at my comment, I wasn't the one that
brought up the fact that "Redhat is trying to play Microsoft", Ulrich was
the one that brought it into the argument ... I was just curious as to how
far they planned on getting to what M$ systems do ...

On Thu, 2002-05-23 at 15:20, Michael Meskes wrote:

On Wed, May 22, 2002 at 10:58:15AM -0700, Ulrich Drepper wrote:

On Wed, 2002-05-22 at 10:51, Lamar Owen wrote:

What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5
(same as Red Hat 7.3's version).

This is a completely different version. Once Debian updates (in a few
years) they'll get the same result.

Ulrich, how shall I understand this? I'm pretty sure Oliver
does not use a Debian 2.2 system with glibc-2.1.3 but a pretty
up-to-date one. The glibc version in the soon to be released Woody
release is 2.2.5.

In fact, I run "unstable" and the version is 2.2.5-6. I couldn't see
any reference in the Debian changelog, but I'm wondering if the change
was commented out. I haven't had the time to download the source and go
looking.

--
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C

"I will praise thee; for I am fearfully and wonderfully
made..." Psalms 139:14

On Thu, May 23, 2002 at 09:29:06AM -0700, Ulrich Drepper wrote:

On Thu, 2002-05-23 at 07:20, Michael Meskes wrote:

The glibc version in the soon to be released Woody
release is 2.2.5.

The version in RHL7.3 is 2.2.5-34. This is not what Debian uses. Maybe
you should read the changelog for the version.

So with your arithmetics 2.2.5 != 2.2.5. Hey that's great. How about
naming the next Linux kernel release 2.0 just to confuse some people?
:-)

Seriously though, while this is offtopic on this list, I wonder what's
going on. If I have a system with glibc 2.2.5, I cannot expect this to
be the same version as on another system with glibc 2.2.5. Is this the
correct understanding?

And which changelog should I read? The RedHat one, the Debian one or the
glibc one?

Or does the -34 mean more than just the RedHat version number? The
Debian version is correctly named 2.2.5-6 where the -6 means that this
is the 6th release of glibc 2.2.5 for Debian,

Michael

--
Michael Meskes
Michael@Fam-Meskes.De
Go SF 49ers! Go Rhein Fire!
Use Debian GNU/Linux! Use PostgreSQL!

...

But anybody using Unix dates as "general dates" has leaped into exactly the
same sort of trap that caused people to get so paranoid about Y2K.

Certainly true. We don't use Unix dates as "general dates", we use the
Unix time zone database and API for dates and times within the year
range of 1903 to 2038. Well, up until now anyway...

Prior to the 1900s, the concept of time zones was more localized and not
universally adopted. In the US, a first round of time zone
standardization came with the transcontinental railroads in the 1860s.
After 2038, it is a good bet that time zones will resemble those in use
today, but they are as much a political construct as a physical one so
the details are subject to change.

- Thomas

On Fri, 24 May 2002 06:10:39 PDT, the world broke into rejoicing as
Thomas Lockhart <lockhart@fourpalms.org> said:

...

But anybody using Unix dates as "general dates" has leaped into exactly the
same sort of trap that caused people to get so paranoid about Y2K.

Certainly true. We don't use Unix dates as "general dates", we use the
Unix time zone database and API for dates and times within the year
range of 1903 to 2038. Well, up until now anyway...

I don't think going past 1970 is particularly safe; it certainly doesn't
seem to fit with ANSI...

By the way, the seemingly relevant link to look at for TZ info is
http://www.twinsun.com/tz/tz-link.htm, linking to the data used by
various Unix implementations.

Prior to the 1900s, the concept of time zones was more localized and
not universally adopted. In the US, a first round of time zone
standardization came with the transcontinental railroads in the 1860s.
After 2038, it is a good bet that time zones will resemble those in
use today, but they are as much a political construct as a physical
one so the details are subject to change.

Some of the zones are quite peculiar if you head to Africa and Asia;
there are some sitting on 15 minute intervales, rather than the usual 1h
intervals.

(The classic Canadian timezone joke is "World ends at 9:00; 9:30 in
Newfoundland". For more information, see TZ='America/St_Johns')
--
(concatenate 'string "chris" "@cbbrowne.com")
http://www.ntlug.org/~cbbrowne/spreadsheets.html
"Heuristics (from the French heure, "hour") limit the amount of time
spent executing something. [When using heuristics] it shouldn't take
longer than an hour to do something."

cbbrowne@cbbrowne.com writes:

By the way, the seemingly relevant link to look at for TZ info is
http://www.twinsun.com/tz/tz-link.htm, linking to the data used by
various Unix implementations.

Oh, this is interesting: it claims that

: This database (often called tz or zoneinfo) is used by several
: implementations, including GNU/Linux, FreeBSD, NetBSD, OpenBSD, DJGPP,
: HP-UX, IRIX, Open UNIX, Solaris, and Tru64.

The actual timezone database seems to consist of about half a meg of
heavily commented text data files. The accompanying code (probably
providing far more functionality than we actually need) is under 400k.
(Both figures are for uncompressed source.) Not large at all.

I cannot find any sign of a copyright or license in the files; I think
it is intended to be public domain, and in any case if the *BSDs are
using it then it must have a BSD-compatible license.

It seems to me that it'd be really practical to just take what we need
out of this distribution and forgo all dependency on system-provided
timezone databases. And, since there's a mailing list maintaining it,
we could expect someone else to handle updates ;-) ... we'd just have
to be careful to use the database files unmodified, so that we could
drop in new releases from time to time.

Comments? Anyone want to do the legwork?

regards, tom lane

Tom Lane wrote:

It seems to me that it'd be really practical to just take what we
need out of this distribution and forgo all dependency on
system-provided timezone databases. And, since there's a mailing
list maintaining it, we could expect someone else to handle updates
;-) ... we'd just have to be careful to use the database files
unmodified, so that we could drop in new releases from time to time.

Comments? Anyone want to do the legwork?

I don't understand precisely what need to be done, but I'll give it a
shot if you get me pointed in the right direction.

<downloads and looks at code>
I see that tzcode2002c.tar.gz includes a mktime() function. Is the idea
to pull this out (with just whatever support it needs), incorporate it
into PostgreSQL source (perhaps in a new src/backend/utils/tz directory)
and use this in place of the system provided mktime()?

Joe

Joe Conway <mail@joeconway.com> writes:

I don't understand precisely what need to be done, but I'll give it a
shot if you get me pointed in the right direction.
<downloads and looks at code>
I see that tzcode2002c.tar.gz includes a mktime() function. Is the idea
to pull this out (with just whatever support it needs), incorporate it
into PostgreSQL source (perhaps in a new src/backend/utils/tz directory)
and use this in place of the system provided mktime()?

Well, that's the zeroth-order approximation. We should take the
opportunity to get out from under the mktime()/tzset() API. The real
idea here is to make use of the timezone database info in the ways that
Postgres needs. Some things that are not good about mktime()/tzset():

* Arbitrary restrictions on range of dates. We certainly don't want to
be limited by a 32-bit time_t, whether you think it's signed or not.
The APIs should be recast in terms of PG's preferred internal
representations. (Lockhart would be the man to point you in the right
direction here, not me.)

* No way to tell whether a user-provided timezone name is actually good.

* No support for concurrent access to multiple zones, short of flushing
all memory of one zone to load the next. Although we do not really need
this now, I can foresee wanting it. I'd be inclined to store all the
info about a particular zone in some struct that can be referenced by
a pointer; that would give us the flexibility to have multiple such
structs floating around a backend in the future (perhaps living in a
hashtable indexed by timezone name?)

My guess is that we want to borrow the parts of the tzcode library that
are associated with reading a tz database file and loading it into some
suitable internal representation; there's probably not a lot else that
we'll want to use as-is.

regards, tom lane

Michael Meskes writes:

Or does the -34 mean more than just the RedHat version number? The
Debian version is correctly named 2.2.5-6 where the -6 means that this
is the 6th release of glibc 2.2.5 for Debian,

Just for general amusement: I run SuSE's glibc 2.2.5-38 which contains
neither the questionable code in the original sources nor is there any
reference to it in the patch set. Go figure.

--
Peter Eisentraut peter_e@gmx.net

On Fri, 2002-05-24 at 12:03, Peter Eisentraut wrote:

Or does the -34 mean more than just the RedHat version number? The
Debian version is correctly named 2.2.5-6 where the -6 means that this
is the 6th release of glibc 2.2.5 for Debian,

Just for general amusement: I run SuSE's glibc 2.2.5-38 which contains
neither the questionable code in the original sources nor is there any
reference to it in the patch set. Go figure.

This is getting silly. Does nobody here understand that the release
number is local for each distribution. Comparing them does not lead to
anything. If you want to find out run

rpm -q --changelog glibc | less

on a RH system. Don't know what other systems provide in this
direction. You'll see that the glibc in RHL7.3 contains a lot of the
code from the glibc 2.3 branch. It's not named 2.2.90 because major
pieces are missing.

If you still don't know that version numbers are meaningless for
determining feature lists you might want to consider going back to your
CS101 class and revisit software configuration management.

--
---------------. ,-. 1325 Chesapeake Terrace
Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA
Red Hat `--' drepper at redhat.com `------------------------

Tom Lane wrote:

Well, that's the zeroth-order approximation. We should take the
opportunity to get out from under the mktime()/tzset() API. The real
idea here is to make use of the timezone database info in the ways that
Postgres needs. Some things that are not good about mktime()/tzset():

* Arbitrary restrictions on range of dates. We certainly don't want to
be limited by a 32-bit time_t, whether you think it's signed or not.
The APIs should be recast in terms of PG's preferred internal
representations. (Lockhart would be the man to point you in the right
direction here, not me.)

* No way to tell whether a user-provided timezone name is actually good.

* No support for concurrent access to multiple zones, short of flushing
all memory of one zone to load the next. Although we do not really need
this now, I can foresee wanting it. I'd be inclined to store all the
info about a particular zone in some struct that can be referenced by
a pointer; that would give us the flexibility to have multiple such
structs floating around a backend in the future (perhaps living in a
hashtable indexed by timezone name?)

My guess is that we want to borrow the parts of the tzcode library that
are associated with reading a tz database file and loading it into some
suitable internal representation; there's probably not a lot else that
we'll want to use as-is.

Well, this does sound a bit more involved than I was envisioning. There
are a few items wrt SRFs that I should finish first, but I'll come back
to this afterward if no one else does first.

Joe

...

Well, this does sound a bit more involved than I was envisioning. There
are a few items wrt SRFs that I should finish first, but I'll come back
to this afterward if no one else does first.

The first cut might be to reproduce the functionality we already have.
That would allow us to (optionally) use this internal code *or* the
system-provided code with a configure-time switch. We could then strip
out two of the three time zone interface styles we support. And we could
(possibly) allow folks to use their built-in time zone databases if they
want, to minimize inconsistancies between PostgreSQL and other programs
on the system. We might need to modify function and variable signatures
to avoid conflicts with system-supplied libraries.

The next step would be to see how to generalize this past Y2038 (as
mentioned previously, time zone info for pre-1900 is not likely to be
interesting). If it involves mass substitution of time_t for, say,
pg_time_t, then that might be all we need for the second phase, at which
time we've broken the y2038 limit by moving to 64-bit integer time.

The last phase could be extending the API to allow multiple simultaneous
time zones, detection of bad time zones, etc etc. This would involve API
changes or extensions, and breaks compatibility with system-supplied
infrastructure.

- Thomas

Thomas Lockhart <lockhart@fourpalms.org> writes:

The last phase could be extending the API to allow multiple simultaneous
time zones, detection of bad time zones, etc etc. This would involve API
changes or extensions, and breaks compatibility with system-supplied
infrastructure.

One thing that wasn't clear to me, but could use investigation: if so
many systems are using the same underlying timezone database info, maybe
there is some commonality at a level below the ISO mktime/tzset/etc API.
If we could make use of the system-provided TZ database at a lower level
while still using our own APIs not tied to time_t, it'd answer the issue
of compatibility with the surrounding system. (Which is a real issue,
I agree --- we should be able to accept the system's standard TZ setting
if possible.)

regards, tom lane

The last phase could be extending the API to allow multiple simultaneous
time zones, detection of bad time zones, etc etc. This would involve API
changes or extensions, and breaks compatibility with system-supplied
infrastructure.

One thing that wasn't clear to me, but could use investigation: if so
many systems are using the same underlying timezone database info, maybe
there is some commonality at a level below the ISO mktime/tzset/etc API.
If we could make use of the system-provided TZ database at a lower level
while still using our own APIs not tied to time_t, it'd answer the issue
of compatibility with the surrounding system. (Which is a real issue,
I agree --- we should be able to accept the system's standard TZ setting
if possible.)

The fundamental problem (which of course can have a fundamental solution
;) is that a time zone database built with a 32-bit time_t will have
time zone info through 2038 only (it is a binary file with 32-bit time
fields -- almost certainly anyway). So if we have an extended time zone
infrastructure using something different for time_t we would need to
handle the case of reading non-extended time zones databases, which puts
us back to having limitations.

I'm guessing that a better approach might be to have our time zone stuff
inside our own API, which then could choose to call, for example,
mktime() or pg_mktime(), which could each have different signatures.
Then the heuristics for matching one to the other are isolated to our
thin API implementation, not to the underlying system- or pg-provided
libraries.

afaik there is no API provision for the "inverse time zone" problem of
matching "stringy time zones" to numeric offsets for input date/times.
The time zone databases themselves don't lend themselves to this, since
the tables have those stringy zones somewhere on the right hand side of
each row of information and the fields can change from year to year.

- Thomas

Thomas Lockhart <lockhart@fourpalms.org> writes:

The fundamental problem (which of course can have a fundamental solution
;) is that a time zone database built with a 32-bit time_t will have
time zone info through 2038 only (it is a binary file with 32-bit time
fields -- almost certainly anyway).

I'm not sure that the time zone database tables store time_t's at all.
They certainly could be coded not to use 'em; but I do not know exactly
how various vendors have chosen to represent the data.

A random extract from the tzdata2002c files looks like:

Zone America/Chicago -5:50:36 - LMT 1883 Nov 18 12:00
-6:00 US C%sT 1920
-6:00 Chicago C%sT 1936 Mar 1 2:00
-5:00 - EST 1936 Nov 15 2:00
-6:00 Chicago C%sT 1942
-6:00 US C%sT 1946
-6:00 Chicago C%sT 1967
-6:00 US C%sT

which might well be represented with separate y/m/d/h/m fields...
certainly we'd choose some such thing if we have to implement it
ourselves.

regards, tom lane

The last phase could be extending the API to allow multiple simultaneous
time zones, detection of bad time zones, etc etc. This would involve API
changes or extensions, and breaks compatibility with system-supplied
infrastructure.

One thing that wasn't clear to me, but could use investigation: if so
many systems are using the same underlying timezone database info, maybe
there is some commonality at a level below the ISO mktime/tzset/etc API.
If we could make use of the system-provided TZ database at a lower level
while still using our own APIs not tied to time_t, it'd answer the issue
of compatibility with the surrounding system. (Which is a real issue,
I agree --- we should be able to accept the system's standard TZ setting
if possible.)

The fundamental problem (which of course can have a fundamental
solution ;) is that a time zone database built with a 32-bit time_t
will have time zone info through 2038 only (it is a binary file with
32-bit time fields -- almost certainly anyway). So if we have an
extended time zone infrastructure using something different for time_t
we would need to handle the case of reading non-extended time zones
databases, which puts us back to having limitations.

Ah, but the database in question _doesn't_ consist of 32 bit time_t
values.

It consists of things like:

# @(#)zone.tab 1.26
#
# TZ zone descriptions
#
# From Paul Eggert <eggert@twinsun.com> (1996-08-05):
#
# This file contains a table with the following columns:
# 1. ISO 3166 2-character country code. See the file `iso3166.tab'.
# 2. Latitude and longitude of the zone's principal location
# in ISO 6709 sign-degrees-minutes-seconds format,
# either +-DDMM+-DDDMM or +-DDMMSS+-DDDMMSS,
# first latitude (+ is north), then longitude (+ is east).
# 3. Zone name used in value of TZ environment variable.
# 4. Comments; present if and only if the country has multiple rows.
#
# Columns are separated by a single tab.
# The table is sorted first by country, then an order within the country that
# (1) makes some geographical sense, and
# (2) puts the most populous zones first, where that does not contradict (1).
#
# Lines beginning with `#' are comments.
#
#country-
#code coordinates TZ comments
AD +4230+00131 Europe/Andorra
AE +2518+05518 Asia/Dubai
AF +3431+06912 Asia/Kabul
AG +1703-06148 America/Antigua
AI +1812-06304 America/Anguilla
AL +4120+01950 Europe/Tirane
AM +4011+04430 Asia/Yerevan
AN +1211-06900 America/Curacao
AO -0848+01314 Africa/Luanda

Then a "leapseconds" table, looking like:
# The correction (+ or -) is made at the given time, so lines
# will typically look like:
# Leap YEAR MON DAY 23:59:60 + R/S
# or
# Leap YEAR MON DAY 23:59:59 - R/S

# If the leapsecond is Rolling (R) the given time is local time
# If the leapsecond is Stationary (S) the given time is UTC

# Leap YEAR MONTH DAY HH:MM:SS CORR R/S
Leap 1972 Jun 30 23:59:60 + S
Leap 1972 Dec 31 23:59:60 + S
Leap 1973 Dec 31 23:59:60 + S
Leap 1974 Dec 31 23:59:60 + S
Leap 1975 Dec 31 23:59:60 + S
Leap 1976 Dec 31 23:59:60 + S

And then a set of rules about timezone adjustments for all sorts of
localities, including the following:

# Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S
# Summer Time Act, 1916
Rule GB-Eire 1916 only - May 21 2:00s 1:00 BST
Rule GB-Eire 1916 only - Oct 1 2:00s 0 GMT
# S.R.&O. 1917, No. 358
Rule GB-Eire 1917 only - Apr 8 2:00s 1:00 BST
Rule GB-Eire 1917 only - Sep 17 2:00s 0 GMT

# Zone NAME GMTOFF RULES FORMAT [UNTIL]
Zone Antarctica/Casey 0 - zzz 1969
8:00 - WST # Western (Aus) Standard Time
Zone Antarctica/Davis 0 - zzz 1957 Jan 13
7:00 - DAVT 1964 Nov # Davis Time
0 - zzz 1969 Feb
7:00 - DAVT
Zone Antarctica/Mawson 0 - zzz 1954 Feb 13
6:00 - MAWT # Mawson Time

I'm guessing that a better approach might be to have our time zone
stuff inside our own API, which then could choose to call, for
example, mktime() or pg_mktime(), which could each have different
signatures. Then the heuristics for matching one to the other are
isolated to our thin API implementation, not to the underlying system-
or pg-provided libraries.

matching "stringy time zones" to numeric offsets for input date/times.
The time zone databases themselves don't lend themselves to this,
since the tables have those stringy zones somewhere on the right hand
side of each row of information and the fields can change from year to
year.

The ultimate goal would seem likely to be to store dates internally in
some form like UTC, with some reasonably huge dynamic range, that is,
not limited to 32 bit timestamps, but rather using something like a
proleptic Gregorian calendar (per _Calendrical Calculations_, page 50).

Some reasonable treatments would include:

- 32 bits is an signed int indicating number of days since GREG_EPOCH,
where logical epochs would include January 1, 1, January 1, 1900, or
perhaps even something actually proleptic (proleptic indicates
"future"), such as January 1, 2038.

- 8 bits indicating the month; 8 bits indicating the day of month;
16 bits providing a range of years from -32767 to 32768.

Both have merits...

Timestamps would then forcibly expand things by _at least_ 22 bits, the
minimum needed to express 1/100ths of seconds. Might as well head on to
32 bits for the time and so have something that can easily represent
values down to well below a millisecond.

The "stringy stuff" indicates how values are to be displayed or parsed.
It does nothing about what is stored internally, or at least shouldn't.
--
(reverse (concatenate 'string "gro.gultn@" "enworbbc"))
http://www.cbbrowne.com/info/emacs.html
In the name of the Lord-High mutant, we sacrifice this suburban girl
-- `Future Schlock'

On Fri, 24 May 2002, Peter Eisentraut wrote:

Michael Meskes writes:

Or does the -34 mean more than just the RedHat version number? The
Debian version is correctly named 2.2.5-6 where the -6 means that this
is the 6th release of glibc 2.2.5 for Debian,

Just for general amusement: I run SuSE's glibc 2.2.5-38 which contains
neither the questionable code in the original sources nor is there any
reference to it in the patch set. Go figure.

You've got to remember that you're talking about systems where, a long time ago
now, certain groups felt it necessary to supply nonstandard versions of the
core component (the kernel). Sure they helped development of the kernel but
only through bastardisation of version numbers where 2.0.1 didn't really mean
a Linux 2.0.1 kernel. Is it really surprising the system support stuff has been
mangled beyond sense?

Anyway, I've composed several and aborted all but this message on this subject
and I'm not going to persue it. I have my own views on the right and wrongs off
the change in glibc but they wouldn't have advanced anything so I'm keeping
quiet on it, still. It seems there is a solution forming. Plus, I'd hate to
side with the baddies from the first paragraph :)

--
Nigel J. Andrews
Director

---
Logictree Systems Limited
Computer Consultants

On Fri, May 24, 2002 at 12:15:47PM -0700, Ulrich Drepper wrote:

This is getting silly. Does nobody here understand that the release

Yes, but I'm not sure on which side.

number is local for each distribution. Comparing them does not lead to

No, this is simply not true. The version number is what the upstream
gives its release. No more no less. What RH does is becoming as subtly
incompatible a possible. If that's the goal, it doesn't look like free
software for me. Sure all changes are published, but why forcing this
kind of difference between linux distributions? Why not calling it 2.2.6
or something if there has to be some changes that are not compatible?

Michael
--
Michael Meskes
Michael@Fam-Meskes.De
Go SF 49ers! Go Rhein Fire!
Use Debian GNU/Linux! Use PostgreSQL!

On Sat, 25 May 2002, Michael Meskes wrote:

No, this is simply not true. The version number is what the upstream
gives its release. No more no less. What RH does is becoming as subtly
incompatible a possible. If that's the goal, it doesn't look like free
software for me. Sure all changes are published, but why forcing this
kind of difference between linux distributions? Why not calling it 2.2.6
or something if there has to be some changes that are not compatible?

Or rlibc? :)

On Friday 24 May 2002 03:15 pm, Ulrich Drepper wrote:

This is getting silly.

Yes, Ulrich, it is. Very silly. And Red Hat's stance is one of the silliest,
IMHO.

You'll see that the glibc in RHL7.3 contains a lot of the
code from the glibc 2.3 branch. It's not named 2.2.90 because major
pieces are missing.

If you still don't know that version numbers are meaningless for
determining feature lists you might want to consider going back to your
CS101 class and revisit software configuration management.

IOW, Red Hat's glibc 2.2.5 isn't really pristine glibc 2.2.5 as found straight
from the GNU repository. In fact, Red Hat glibc 2.2.5 isn't really 2.2.5 --
how about 2.2.96? :-) .96 was good enough for gcc....

Furthermore, Red Hat glibc 2.2.5 isn't even fully compatible with GNU glibc
2.2.5 -- at least in the area of time_t stuff.

In the open source world, version numbers are actually supposed to mean
something -- at least for package dependencies. Of course, I also have read
the kernel-2.4.18 source RPM and its 21.8MB 'ac-bits' patch.

You do realize that this sort of thing doesn't help Red Hat's PR state amongst
the greater open source community, right? Nor would it help Mandrake, SuSE,
or any other Linux distributor (I specifically excluded Debian due to its
unique community supported state). But, if you don't care about the greater
open source community, well...

And I say all of that while running and enjoying the greater part of Red Hat
7.3. For the most part it is extraordinarily stable. And I know that that
21.8MB kernel patch is one of the reasons it is so stable. But I still
question the versioning of glibc.

So, in summary, the glibc version number in any particular linux distribution
is meaningless because the distributor is free to patch the bloody daylights
out of it at any time. Sweet. And so standard.

But, if glibc 2.3 is where this bit came from, it is just a matter of time
before all Linux distributions (that aren't willing to patch away) get this
braindead behavior. Oh well. The general solution will happen.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Tom Lane wrote:

Thomas Lockhart <lockhart@fourpalms.org> writes:

Why should we rely on broken glibc and the standard? Why don't we make
our own mktime() and use it on all platforms.

The downside to doing that is that we then take over maintenance of the
code and, more importantly, the timezone database.

But it might be the best thing to do.

I've been sorta thinking the same thing. We could get out from under
the Y2038 issue, and also eliminate a whole lot of platform
dependencies. Not to mention sillinesses like being unable to recognize
a bad timezone name when it's fed to us.

Exactly how much work (and code bulk) would we be taking on? I've
never looked at how big the timezone databases are...

I am not really excited about distributing a timezone database as part
of PostgreSQL, and it wouldn't match the OS's timezone. (We do need a
64-time time_t, but I think we can wait to get closer to 2038.) Can we
detect if glibc is being used for the compile (easy), and substitute a
non-broken mktime in the link path ahead of glibc's mktime? Seems that
would be the easiest solution.

Of course, pre-1970 dates then wouldn't match the OS on glibc systems,
but that seems like a win. :-)

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Trond Eivind Glomsr���d wrote:

On Tue, 21 May 2002, Lamar Owen wrote:

On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote:

I see. This behavior is consistent with the fact that mktime is
supposed to return -1 on error, but then is broken in every other Unix
implementation that I know.

Any other workaround than downgrade or install FreeBSD?

Complain to Red Hat. Loudly. However, as this is a glibc change, other
distributors are very likely to fold in this change sooner rather than
later.

Relying on nonstandardized/nondocumented behaviour is a program bug, not a
glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
it, but glibc is not the component with a problem.

No one has really answered the question --- if the way PostgreSQL is
using mktime() for pre-1970 dates is wrong, why do timezone databases
have pre-1970 timezone information?

I assume Linux does or the old mktime() wouldn't have worked for
pre-1970 dates.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026