Psql 7.2.1 Regress tests failed on RedHat 7.3

Denis Gasparin <denis@edistar.com> writes:

Is there any reason? Any tip?

The diffs seem to be in the same places as for Solaris --- perhaps
someone tried to correct 7.3's timezone database for the 1947 DST
rules? (AFAIK, Solaris is more correct for 1947 than most other
timezone databases.) If so, they blew it ...

regards, tom lane

Denis Gasparin <denis@edistar.com> writes:

I have tried to install postgresql 7.2.1 on the new Red Hat 7.3. The
regression test failed in three cases:

The tests are buggy in some cases... the expected results are not
locale aware (e.g. "12.42" vs "12,42", "A C b" v "A b C")

--
Trond Eivind Glomsr�d
Red Hat, Inc.

I've tried to replace the RH 7.3 timezone database (in
/usr/share/zoneinfo) with that of RH 7.2 (where regress tests work).

The problem persists... In RH 7.3 there is also a new version of the
glibc... (2.2.5 - 34)

If anyone has any idea...

--
Doct. Eng. Denis Gasparin: denis@edistar.com
---------------------------
Programmer & System Administrator - Edistar srl

Il gio, 2002-05-09 alle 19:39, Tom Lane ha scritto:

Denis Gasparin <denis@edistar.com> writes:

I have tried to install postgresql 7.2.1 on the new Red Hat 7.3. The
regression test failed in three cases:

The tests are buggy in some cases... the expected results are not
locale aware (e.g. "12.42" vs "12,42", "A C b" v "A b C")

This isn't the case you've indicated... This is the regression test diff
output:

*** ./expected/abstime.out      Wed Nov 21 19:27:25 2001
--- ./results/abstime.out       Thu May  9 17:27:00 2002
***************
*** 44,50 ****
         | Wed Dec 31 16:00:00 1969 PST
         | infinity
         | -infinity
!        | Sat May 10 23:59:12 1947 PST
         | invalid
  (7 rows)

--- 44,50 ----
         | Wed Dec 31 16:00:00 1969 PST
         | infinity
         | -infinity
!        | Sat May 10 15:59:12 1947 PST
         | invalid
  (7 rows)

As you can see the time is different not the time/date format...

--
Doct. Eng. Denis Gasparin: denis@edistar.com
---------------------------
Programmer & System Administrator - Edistar srl

...

This isn't the case you've indicated... This is the regression test diff
output:
*** ./expected/abstime.out      Wed Nov 21 19:27:25 2001
--- ./results/abstime.out       Thu May  9 17:27:00 2002
!        | Sat May 10 23:59:12 1947 PST

...

! | Sat May 10 15:59:12 1947 PST

...

As you can see the time is different not the time/date format...

Ooh. Maybe the SuSE folks who were working on glibc actually changed it
the way they were talking about. Here (may be) the issue:

Posix demands that mktime() and friends return -1 on error or on "unable
to convert time". Traditionally, many implementations (AIX was a notable
exception, adding to its rep as a brain-damaged system) ignored that
restriction, willingly converting for times before 1970, and returning a
negative number. And timezone databases supported times before 1970, and
the world was happy.

But that Posix requirement made it difficult to support the time "one
second before 1970", which happens to be "-1". So the idea was to start
returning -1 for any time before 1970. That sucks. But for PostgreSQL,
we actually don't use the results of mktime(), but we *do* use a
side-effect of the call, which fills in the time zone information (and a
few other fields) in the structures given to mktime() as *input*. We
currently ignore the status return of mktime(), since we guard against
illegal calling parameters another way.

I would be very happy if the implementation of mktime() continued to
*properly* fill in these fields, even though it insists on returning an
error condition for dates before 1970. As it is (assuming that a change
actually has occurred) the timezone system in GNU systems has been badly
and gratuitously damaged in the newest releases of glibc.

Any suggestions on how to pursue this??

- Thomas

In article <3CDBD134.15FCF31D@fourpalms.org>,
Thomas Lockhart <lockhart@fourpalms.org> wrote:

error condition for dates before 1970. As it is (assuming that a change
actually has occurred) the timezone system in GNU systems has been badly
and gratuitously damaged in the newest releases of glibc.

The mktime.c for glibc hasn't changed since Jul 5, 2001.

mrc

--
Mike Castle dalgoda@ix.netcom.com www.netcom.com/~dalgoda/
We are all of us living in the shadow of Manhattan. -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

error condition for dates before 1970. As it is (assuming that a change
actually has occurred) the timezone system in GNU systems has been badly
and gratuitously damaged in the newest releases of glibc.

The mktime.c for glibc hasn't changed since Jul 5, 2001.

There is a thread (I'm pretty sure it was on-list) which points at a
change log which shows that the behavior of mktime() was changed in the
glibc development tree. My recollection might be wrong on details, but I
strongly recall this to be substantially correct.

Hopefully the changes are backed out or at least the "set the time zone"
side effect continues to be supported (it wasn't clear what the actual
change in implementation would do; I didn't look at the source).
Otherwise, Linux systems will look more and more like they were built by
kids who weren't even alive before 1970. Hmm, maybe that's the
explanation... ;)

- Thomas (*definitely* pre-1970)

On Friday 10 May 2002 09:55 am, Thomas Lockhart wrote:

Ooh. Maybe the SuSE folks who were working on glibc actually changed it
the way they were talking about. Here (may be) the issue:

But that Posix requirement made it difficult to support the time "one
second before 1970", which happens to be "-1". So the idea was to start
returning -1 for any time before 1970. That sucks. But for PostgreSQL,
we actually don't use the results of mktime(), but we *do* use a
side-effect of the call, which fills in the time zone information (and a
few other fields) in the structures given to mktime() as *input*. We
currently ignore the status return of mktime(), since we guard against
illegal calling parameters another way.

I would be very happy if the implementation of mktime() continued to
*properly* fill in these fields, even though it insists on returning an
error condition for dates before 1970. As it is (assuming that a change
actually has occurred) the timezone system in GNU systems has been badly
and gratuitously damaged in the newest releases of glibc.

Well, I went to bat for this a little bit ago, relating to a bug report, but
I've struck out. The ISO C standard spells it out plainly that dates before
1970 are just simply illegal for mktime and friends. So, glibc was aligned
with the ISO standard (available online -- see my link in the post regarding
the mktime bug in Red Hat 7.3). This will change ALL newer Linux
distributions, and thus we need to correct our usage, as the glibc people are
not likely to change -- in fact, they will likely say that our code is
broken.

Any suggestions on how to pursue this??

What information do we need?

Frankly, relying on a side-effect of a call in this way is our bug. Sorry.
It was a neat hack, but it now is broken. Where is this in the tree? I can
take a look at it if no one else wants to.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen <lamar.owen@wgcr.org> writes:

Well, I went to bat for this a little bit ago, relating to a bug report, but
I've struck out. The ISO C standard spells it out plainly that dates before
1970 are just simply illegal for mktime and friends.

Well, since glibc apparently has no higher ambition than to work for
post-1970 dates, we may have little choice but to throw out mktime and
implement our own timezone library. Ugh. It is pretty damn annoying
that they aren't interested in fixing their problem...

regards, tom lane

[HACKERS added to cc:, GENERAL dropped]
On Monday 20 May 2002 11:39 pm, Tom Lane wrote:

Lamar Owen <lamar.owen@wgcr.org> writes:

Well, I went to bat for this a little bit ago, relating to a bug report,
but I've struck out. The ISO C standard spells it out plainly that dates
before 1970 are just simply illegal for mktime and friends.

Well, since glibc apparently has no higher ambition than to work for
post-1970 dates, we may have little choice but to throw out mktime and
implement our own timezone library. Ugh. It is pretty damn annoying
that they aren't interested in fixing their problem...

They are just wanting to be standard. I know this; I just can't say how I
know this. But the link to the ISO definition is
http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap04.html#tag_04_14
FWIW.

While I don't agree with the standard, trying to be standard isn't really a
'problem'. Relying on a side-effect of a nonstandard call is the problem.

Can we pull in the BSD C library's mktime()? OR otherwise utilize it to fit
this bill?

Looking at src/backend/utils/adt/datetime.c indicates that it might not be too
difficult. It was WISE to centralize the use of mktime in the one function,
it appears.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

On Tuesday 21 May 2002 09:23 am, Thomas Lockhart wrote:

While I don't agree with the standard, trying to be standard isn't really
a 'problem'. Relying on a side-effect of a nonstandard call is the
problem.

In my mind no one associated with glibc gets high marks for anything
having to do with this change. It is unbelievably short sighted.

Oh, I most certainly agree with you on this Thomas. The glibc people are just
being adamant about it being 'standard.' And I certainly didn't mean to step
on your toes, either -- as I can tell this is a sore point for you. I'm just
trying to see what, if anything, we can do about it.

Can we pull in the BSD C library's mktime()? OR otherwise utilize it to
fit this bill?

Maybe, but probably forcing a *really* annoying code fork or patch to
get the entry points to play nice with glibc. We'll also need to figure
out how to manage the time zone database and how to keep it in sync.

This is a seriously big problem, and we'll need to research what to do
next. One possibility is to strip out all string time zone behavior and
support only the SQL date/time standard, which specifies only numeric
offsets and ignores real-world time zone usage and behaviors. Hmm, IBM
contributed to that standard too, maybe the common thread is not a
coincidence.

Well, the existing behavior, according to my first read of the code, is to
assume UTC if the time_t is predicted to be out of range. There is a macro
for this, I see. And the problem is that the out-of-range condition is
happening at a different place. I don't like this thought, but the most
consistent, least-common-denominator tack would to be flag anything prior to
epoch as out-of-range, even if the underlying calls can handle negative
time_t. I don't like that one whit. But I like inconsistent data even less.

The new glibc behavior is really disappointing. Help and ideas are
appreciated; reimplementing an entire date/time system may be a lot of
work.

Well, it was good foresight on your part to put all the mktime stuff in the
one place. I'm going to go through it and see if I understand what I'm
looking at first.

But I see a couple of possibilities that we can control:
1.) Have configure test for broken mktime and sub our own mktime in that case
(if this is even possible -- while the various BSD's have mktime and friends,
how difficult is it going to be to unshackle that from a BSD kernel
underneath -- I've not looked at the code for OpenBSD's mktime (which I have
on hand), but I guess I will take a look now);
2.) Rewrite our stuff to not depend on any mktime, and thus be more portable
(perhaps?).

But, in any case, I didn't mean to step on your toes by any of my comments; I
completely agree with you that glibc and the ISO C standard cited are daft in
this.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

...

But, in any case, I didn't mean to step on your toes by any of my comments; I
completely agree with you that glibc and the ISO C standard cited are daft in
this.

No complaints from my toes; I was just ventilating about stupid
breakage.

- Thomas

In article <17682.1021952389@sss.pgh.pa.us>,
Tom Lane <tgl@sss.pgh.pa.us> wrote:

Well, since glibc apparently has no higher ambition than to work for
post-1970 dates, we may have little choice but to throw out mktime and
implement our own timezone library. Ugh. It is pretty damn annoying
that they aren't interested in fixing their problem...

Ummm... I'm no great admirer of Drepper, but exactly why is this a glibc
problem?

This was clearly an undocumented side-effect that, by pure chance, worked
well until now. So it happens to be glibc that first trips up this issue,
but it could very well have been any mktime implementation.

I think it's unfair to blame glibc. At least until some other standard
comes into existence that makes glibc's implementation invalid.

mrc
--
Mike Castle dalgoda@ix.netcom.com www.netcom.com/~dalgoda/
We are all of us living in the shadow of Manhattan. -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

dalgoda@ix.netcom.com (Mike Castle) writes:

I think it's unfair to blame glibc. At least until some other standard
comes into existence that makes glibc's implementation invalid.

Nonsense. Are you saying it's a mistake that the Linux timezone
databases (and those of most every other Unix, other than maybe AIX)
cover pre-1970 years? The folks who took the trouble to develop
those databases did not think that Unix time began on 1970-01-01.

The glibc guys have decided that they will hew to the letter of a
lowest-common-denominator standard by removing functionality,
namely the ability to do anything with pre-1970 dates.

Since they evidently value a narrow reading of a spec over
functionality, I cannot debate with them. But I think it's a seriously
misguided position. glibc will join AIX as one of only two libc's on
the planet that do not have this functionality. Meanwhile, the Postgres
project will be wasting manpower on reinventing a wheel we shouldn't
have to reinvent.

regards, tom lane