Redhat 7.3 time manipulation bug

On Saturday 18 May 2002 12:34 am, Manuel Sugawara wrote:

Something is pretty broken in redhat 7.3 but I'm not sure what and I
don't have time to dig further

Both cases works correctly in Redhat 7.2. Sorry if this is not the
correct forum however an alert is nice for people planning an Redhat
upgrade. I hope to see pretty soon an update since I don't want to
downgrade my server.

Filed on Red Hat's Bugzilla system as bug# 65227, as I can reliably reporoduce
this bug here, and PostgreSQL 7.2.1 on Red Hat 6.2 on SPARC does not exhibit
the bug.

Tom (or Thomas):

Where would we go to ferret out the source of this bug? More to the point: we
need a test case in C that could expose this as a glibc bug. Methinks Red
Hat would want this bug ferretted out, as it would likely cause problems with
RedHat Database on RH 7.3's glibc.....
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen <lamar.owen@wgcr.org> writes:

Filed on Red Hat's Bugzilla system as bug# 65227, as I can reliably
reporoduce this bug here, and PostgreSQL 7.2.1 on Red Hat 6.2 on
SPARC does not exhibit the bug.

Thanks for filling that report. I couldn't remember what had forgotten
;-)

Regards,
Manuel.

Lamar Owen <lamar.owen@wgcr.org> writes:

Where would we go to ferret out the source of this bug? More to the
point: we need a test case in C that could expose this as a glibc
bug.

Seems like mktime(3) is having problems with dates before the
epoch. Attached is the a program to test this. The glibc source is now
downloading I will try to hunt down this bug but not until the next
week.

Regards,
Manuel.

On Monday 20 May 2002 08:08 pm, Manuel Sugawara wrote:

Where would we go to ferret out the source of this bug? More to the
point: we need a test case in C that could expose this as a glibc
bug.

Seems like mktime(3) is having problems with dates before the
epoch. Attached is the a program to test this. The glibc source is now
downloading I will try to hunt down this bug but not until the next
week.

It's not a bug. At least not according to the ISO C standard. See
http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap04.html#tag_04_14
for the definition of 'Seconds Since the Epoch', then cross-reference to the
man page of mktime.

I don't like it any more than you do, but that is the letter of the standard.

Thomas, any comments?

Our implementation is broken, then. Thomas, is this fixable for a 7.2.x
release, or something for 7.3?
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

...

Our implementation is broken, then. Thomas, is this fixable for a 7.2.x
release, or something for 7.3?

"Our implementation is broken, then" is really not a conclusion to be
reached. The de facto behavior of mktime() on all world-class
implementations is to support pre-1970 times. This has been true forever
afaik, certainly far longer than PostgreSQL (or Postgres) has been in
existance.

Any standard which chooses to ignore pre-1970 dates is fundamentally
broken imho, and I'm really ticked off that the glibc folks have so
glibly introduced a change of this nature and magnitude without further
discussion.

- Thomas

Lamar Owen <lamar.owen@wgcr.org> writes:

On Monday 20 May 2002 08:08 pm, Manuel Sugawara wrote:

Where would we go to ferret out the source of this bug? More to the
point: we need a test case in C that could expose this as a glibc
bug.

Seems like mktime(3) is having problems with dates before the
epoch. Attached is the a program to test this. The glibc source is now
downloading I will try to hunt down this bug but not until the next
week.

It's not a bug. At least not according to the ISO C standard. See
http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap04.html#tag_04_14
for the definition of 'Seconds Since the Epoch', then
cross-reference to the man page of mktime.

I see. This behavior is consistent with the fact that mktime is
supposed to return -1 on error, but then is broken in every other Unix
implementation that I know.

Any other workaround than downgrade or install FreeBSD?

Regards,
Manuel.

On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote:

I see. This behavior is consistent with the fact that mktime is
supposed to return -1 on error, but then is broken in every other Unix
implementation that I know.

Any other workaround than downgrade or install FreeBSD?

Complain to Red Hat. Loudly. However, as this is a glibc change, other
distributors are very likely to fold in this change sooner rather than later.

Try using timestamp without timezone?
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

On Tue, 21 May 2002, Lamar Owen wrote:

On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote:

I see. This behavior is consistent with the fact that mktime is
supposed to return -1 on error, but then is broken in every other Unix
implementation that I know.

Any other workaround than downgrade or install FreeBSD?

Complain to Red Hat. Loudly. However, as this is a glibc change, other
distributors are very likely to fold in this change sooner rather than
later.

Relying on nonstandardized/nondocumented behaviour is a program bug, not a
glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
it, but glibc is not the component with a problem.

--
Trond Eivind Glomsr�d
Red Hat, Inc.

On Tuesday 21 May 2002 12:31 pm, Trond Eivind Glomsr�d wrote:

On Tue, 21 May 2002, Lamar Owen wrote:

However, as this is a glibc change, other
distributors are very likely to fold in this change sooner rather than
later.

Relying on nonstandardized/nondocumented behaviour is a program bug, not a
glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
it, but glibc is not the component with a problem.

In your opinion. Not everyone agrees with the new glibc behavior. In fact,
some here are rather livid about it. But that's a digression. The matter at
hand is making it work right again.

It seems to me someone should have thought about this before making the glibc
change that had the potential for breaking a large application -- regardless
of who is at fault, it's egg in Red Hat's face for not catching it sooner
(and egg in my face as well, as I must admit that I of all people should have
caught this earlier).

Is the change in glibc behavior noted in the release notes? The man page
isn't changed either, from Red Hat 6.2, in fact. The only change is adhering
to the ISO definition of 'Seconds Since the Epoch' rather than the defacto
industry-accepted definition that has been with us a very long time.

Like PostgreSQL's refusal to upgrade in a sane manner, this should have
received release notes billing, IMHO. Then I, nor anyone else, could
reasonably complain.

But this does show the need for the regression testing packge, no? :-) And it
also shows the danger in becoming too familiar with certain regression tests
failing due to locale -- to the extent that a locale issue was the first
thing thought of. To the extent that I'm going to change my build process to
include regression testing as a part of the build -- and any failure will
abort the build. Maybe that will get my attention. And anyone else's
rebuilding from the source RPM. SuSE already does this. I wonder how
they've handled this issue with 8.0?

In any case, this isn't just a Red Hat problem, as it's going to cause
problems with the use of timestamps on ANY glibc 2.2.5 dist. That's more
than Red Hat, by a large margin.

And I think that it is naive to think that PostgreSQL is the only program that
has used mktime's behavior in the negative-time_t zone. Other packages are
likely impacted, to a greater or lesser extent.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

On Tue, 2002-05-21 at 21:31, Trond Eivind Glomsrød wrote:

On Tue, 21 May 2002, Lamar Owen wrote:

On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote:

I see. This behavior is consistent with the fact that mktime is
supposed to return -1 on error, but then is broken in every other Unix
implementation that I know.

Any other workaround than downgrade or install FreeBSD?

Complain to Red Hat. Loudly. However, as this is a glibc change, other
distributors are very likely to fold in this change sooner rather than
later.

Relying on nonstandardized/nondocumented behaviour is a program bug, not a
glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
it, but glibc is not the component with a problem.

Still it seems kind of silly to have a function that works different
from all other implementations and forces people to use their own
function of the same name (lifted from BSD and also compliant).

Speaking of nonstandardized/nondocumented behaviour, I read from "The
Open Sources" book that if you implement TCP/IP strictly from the RFCs
then it won't interoperate with any other TCP/IP stack.

I hope that Red Hat is not going to be "standards compliant" here ;)

--------------
Hannu

Trond Eivind Glomsrød <teg@redhat.com> writes:

Relying on nonstandardized/nondocumented behaviour is a program bug,
not a glibc bug.

The question is: how this thing didn't show up before? ISTM that
someone is not doing his work correctly.

PostgreSQL needs fixing.

Arguably, however, right now is *a lot easier* to fix glibc, and it's
really needed for production systems using postgreSQL and working on
RedHat. But redhat users doesn't matter, the most important thing is
*strict* conformace to standars, right?

Since we ship both, we're looking at it, but glibc is not the

^^^^^^^^^^^^^^^^^^^
The sad true is: you only answered when the 'Complain to Red Hat'
statement appeared, not a single word before and not a single word
when the bug report were closed. I'm really disappointed.

The nice thing is: glibc is free software and we don't have to wait or
relay on some of the redhat staff members (thanks god) for this to get
fixed or say: for the standard to get extended again. The patch to
glibc is pretty straightforward and attached.

Regards,
Manuel.

On 21 May 2002, Manuel Sugawara wrote:

Trond Eivind Glomsr�d <teg@redhat.com> writes:

Relying on nonstandardized/nondocumented behaviour is a program bug,
not a glibc bug.

The question is: how this thing didn't show up before? ISTM that
someone is not doing his work correctly.

FWIW, I ran the regressions tests some time ago(probably before that
change to glibc) . Since the tests are known
to be broken wrt. time issues anyway (as well as currency, math and sorting),
it's easy to overlook.

PostgreSQL needs fixing.

Arguably, however, right now is *a lot easier* to fix glibc, and it's
really needed for production systems using postgreSQL and working on
RedHat.

You're not "fixing" glibc, you're reintroducing non-standardized, upstream
removed behaviour. That's typically a very bad thing. If anything, it
demonstrates the importance of not using or relying on
unstandardized/undocumented behaviour (and given that time_t is pretty
restrictive anyway, you'll need something else to keep dates. It doesn't
even cover all living people, and definitely not historical dates).

Since we ship both, we're looking at it, but glibc is not the

^^^^^^^^^^^^^^^^^^^
The sad true is: you only answered when the 'Complain to Red Hat'
statement appeared, not a single word before and not a single word
when the bug report were closed. I'm really disappointed.

The bug wasn't open for long, and was closed by someone else.

The nice thing is: glibc is free software

Also, notice that this was where the fix came from: The upstream
maintainers (some of whom work for us, others don't).

--
Trond Eivind Glomsr�d
Red Hat, Inc.

On Tuesday 21 May 2002 03:09 pm, Trond Eivind Glomsr�d wrote:

FWIW, I ran the regressions tests some time ago(probably before that
change to glibc) . Since the tests are known
to be broken wrt. time issues anyway (as well as currency, math and
sorting), it's easy to overlook.

The time tests have never broken in this manner before on Red Hat. When the
original regression failure report was posted, I saw right away that this was
not the run of the mill locale issue -- this was a real problem. Regression
testing must become a regularly scheduled activity, methinks. In the RPM
build process, we can control the locale to the extent that the tests will
pass (except on DST days) reliably. I am going to implement this for my next
RPM set. Along with a patch to this problem -- we _can_ patch around this, I
believe, but it's not likely going to be an easy one.

We have gotten blind to the regular locale-induced failures -- this is not a
good thing.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

=?ISO-8859-1?Q?Trond_Eivind_Glomsr=F8d?= <teg@redhat.com> writes:

Relying on nonstandardized/nondocumented behaviour is a program bug, not a
glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
it, but glibc is not the component with a problem.

A library that can no longer cope with dates before 1970 is NOT my idea
of a component without a problem. We will be looking at ways to get
around glibc's breakage at the application level, since we have little
alternative other than to declare Linux an unsupported platform;
but it's still glibc (and the ISO spec:-() that are broken.

regards, tom lane

On Tue, 2002-05-21 at 18:24, Lamar Owen wrote:

In any case, this isn't just a Red Hat problem, as it's going to cause
problems with the use of timestamps on ANY glibc 2.2.5 dist. That's more
than Red Hat, by a large margin.

I'm running glibc 2.2.5 on Debian and all regression tests pass OK (with
make check). I don't see any note in the glibc Debian changelog about
reversing an upstream change to mktime().

I missed the first messages in this thread and I can't find them in the
archive. What should I be looking for to see if I have the problem you
have encountered or to see why I don't have it if I ought to have?

--
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C

"O come, let us worship and bow down; let us kneel
before the LORD our maker." Psalms 95:6

On Tuesday 21 May 2002 06:09 pm, Oliver Elphick wrote:

On Tue, 2002-05-21 at 18:24, Lamar Owen wrote:

In any case, this isn't just a Red Hat problem, as it's going to cause
problems with the use of timestamps on ANY glibc 2.2.5 dist. That's more
than Red Hat, by a large margin.

I'm running glibc 2.2.5 on Debian and all regression tests pass OK (with
make check). I don't see any note in the glibc Debian changelog about
reversing an upstream change to mktime().

I missed the first messages in this thread and I can't find them in the
archive. What should I be looking for to see if I have the problem you
have encountered or to see why I don't have it if I ought to have?

Hmmm. Compile and run the attached program. If you get -1, it's the new
behavior. It might be interesting to see the differences here.....
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Manuel Sugawara <masm@fciencias.unam.mx> writes:

+#if 0
/* Only years after 1970 are defined.
If year is 69, it might still be representable due to
timezone differnces. */
if (year < 69)
return -1;
+#endif

Hm. If that fixes it, it implies that all the other support for
pre-1970 dates is still there (notably, entries in the timezone tables).

Should we assume that future glibc releases will rip out the timezone
database entries and other support for pre-1970 dates? Or is the
breakage going to stop with mktime?

regards, tom lane

Lamar Owen writes:

SuSE already does this. I wonder how they've handled this issue with
8.0?

Their glibc doesn't have that problem.

Personally, I think if you need time (zone) support before 1970, obtain
one of the various operating systems that support it. There's little
value in hacking around it in PostgreSQL, since the rest of your system
will be broken as well.

--
Peter Eisentraut peter_e@gmx.net

SuSE already does this. I wonder how they've handled this issue with
8.0?

Their glibc doesn't have that problem.

My strong recollection is that a SuSE guy was the one applying the
change. So this is coming to those systems too. I may not remember that
correctly though...

Personally, I think if you need time (zone) support before 1970, obtain
one of the various operating systems that support it. There's little
value in hacking around it in PostgreSQL, since the rest of your system
will be broken as well.

Yes, I'm afraid I agree. In practice, maybe most applications won't
notice. But after getting the Linux time zone databases set up to be
better than most (Solaris has the best I've found for fidelity to
pre-1970 year-to-year conventions) throwing that work away is just plain
silly. I consider this a major gaff on the part of the commercial Linux
houses to not see this coming and to contribute to a better solution.

- Thomas