Regression tests fail with tzdata 2024b

Wolfgang Walther <walther@technowledgy.de> writes:

Building --with-system-tzdata and the latest tzdata 2024b fails the
regression tests for me (see attached .diffs). This seems to be because
of [1], which changed the way "PST8PDT" is handled. This is the timezone
that the regression tests are run with.

That's quite annoying, especially since it was not mentioned in the
2024b release notes. (I had read the notes and concluded that 2024b
didn't require any immediate attention on our part.)

From 2024b on, "PST8PDT" is the same as "America/Los_Angeles", so by
changing the regression tests to use the latter as the default, we're
getting consistent output on at least 2024a and 2024b.

I'm fairly un-thrilled with this answer, not least because it exposes
that zone's idiosyncratic "LMT" offset of -7:52:58 for years before
1883. (I'm surprised that that seems to affect only one or two
regression results.) Also, as a real place to a greater extent
than "PST8PDT" is, it's more subject to historical revisionism when
somebody turns up evidence of local law having been different than
TZDB currently thinks.

We may not have a lot of choice though. I experimented with using
full POSIX notation, that is "PST8PDT,M3.2.0,M11.1.0", but that is
actually worse in terms of the number of test diffs, since it doesn't
match the DST transition dates that the tests expect for years before
2007. Another objection is that the tests would then not exercise any
of the mainstream tzdb-file-reading code paths within the timezone
code itself.

Grumble.

regards, tom lane

Tom Lane:

Also, as a real place to a greater extent
than "PST8PDT" is, it's more subject to historical revisionism when
somebody turns up evidence of local law having been different than
TZDB currently thinks.

I now tried all versions of tzdata which we had in tree back to 2018g,
they all work fine with the same regression test output. 2018g was an
arbitrary cutoff, I just didn't try any further.

In the end, we don't need a default timezone that will never change. We
just need one that didn't change in a reasonable number of releases
going backwards. Once America/Los_Angeles is changed, we need to switch
to a different zone, which could be one that wouldn't work today. Kind
of a sliding window.

One positive might be: With this timezone, we are more likely to see
relevant changes mentioned in the upstream release notes.

Best,

Wolfgang

Wolfgang Walther <walther@technowledgy.de> writes:

Tom Lane:

Also, as a real place to a greater extent
than "PST8PDT" is, it's more subject to historical revisionism when
somebody turns up evidence of local law having been different than
TZDB currently thinks.

I now tried all versions of tzdata which we had in tree back to 2018g,
they all work fine with the same regression test output. 2018g was an
arbitrary cutoff, I just didn't try any further.

Yeah, my belly-aching above is just about hypothetical future
instability. In reality, I'm sure America/Los_Angeles is pretty
well researched and so it's unlikely that there will be unexpected
changes in its zone history.

In the end, we don't need a default timezone that will never change.

We do, really. For example, there's a nonzero chance the USA will
cancel DST altogether at some future time. (This would be driven by
politicians who don't remember the last time, but there's no shortage
of those.) That'd likely break some of our test results, and even if
it happened not to, it'd still be bad because we'd probably lose some
coverage of the DST-transition-related code paths in src/timezone/.
So I'd really be way happier with a test timezone that never changes
but does include DST behavior. I thought PST8PDT fit those
requirements pretty well, and I'm annoyed at Eggert for having tossed
it overboard for no benefit whatever. But I don't run tzdb, so
here we are.

We just need one that didn't change in a reasonable number of
releases going backwards.

We've had this sort of fire-drill before, e.g. commit 8d7af8fbe.
It's no fun, and the potential surface area for unexpected changes
is now much greater than the few tests affected by that change.

But here we are, so I pushed your patch with a couple of other
cosmetic bits. There are still a couple of references to PST8PDT
in the tree, but they don't appear to care what the actual meaning
of that zone is, so I left them be.

regards, tom lane

On Mon, Sep 16, 2024 at 7:09 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Wolfgang Walther <walther@technowledgy.de> writes:

Tom Lane:

Also, as a real place to a greater extent
than "PST8PDT" is, it's more subject to historical revisionism when
somebody turns up evidence of local law having been different than
TZDB currently thinks.

I now tried all versions of tzdata which we had in tree back to 2018g,
they all work fine with the same regression test output. 2018g was an
arbitrary cutoff, I just didn't try any further.

Yeah, my belly-aching above is just about hypothetical future
instability. In reality, I'm sure America/Los_Angeles is pretty
well researched and so it's unlikely that there will be unexpected
changes in its zone history.

In the end, we don't need a default timezone that will never change.

We do, really. For example, there's a nonzero chance the USA will
cancel DST altogether at some future time. (This would be driven by
politicians who don't remember the last time, but there's no shortage
of those.) That'd likely break some of our test results, and even if
it happened not to, it'd still be bad because we'd probably lose some
coverage of the DST-transition-related code paths in src/timezone/.
So I'd really be way happier with a test timezone that never changes
but does include DST behavior. I thought PST8PDT fit those
requirements pretty well, and I'm annoyed at Eggert for having tossed
it overboard for no benefit whatever. But I don't run tzdb, so
here we are.

We just need one that didn't change in a reasonable number of
releases going backwards.

We've had this sort of fire-drill before, e.g. commit 8d7af8fbe.
It's no fun, and the potential surface area for unexpected changes
is now much greater than the few tests affected by that change.

But here we are, so I pushed your patch with a couple of other
cosmetic bits. There are still a couple of references to PST8PDT
in the tree, but they don't appear to care what the actual meaning
of that zone is, so I left them be.

This is an unfortunate change as this will break extensions tests using
pg_regress for testing. We run our tests against multiple minor versions
and this getting backported means our tests will fail with the next minor
pg release. Is there a workaround available to make the timezone for
pg_regress configurable without going into every test?

Regards, Sven Klemm

Sven Klemm <sven@timescale.com> writes:

This is an unfortunate change as this will break extensions tests using
pg_regress for testing. We run our tests against multiple minor versions
and this getting backported means our tests will fail with the next minor
pg release. Is there a workaround available to make the timezone for
pg_regress configurable without going into every test?

Configurable to what? If your test cases are dependent on the
historical behavior of PST8PDT, you're out of luck, because that
simply isn't available anymore (or won't be once 2024b reaches
your platform, anyway).

regards, tom lane

On Mon, Sep 16, 2024 at 5:19 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Configurable to what? If your test cases are dependent on the
historical behavior of PST8PDT, you're out of luck, because that
simply isn't available anymore (or won't be once 2024b reaches
your platform, anyway).

I was wondering whether the timezone used by pg_regress could be made
configurable.

--
Regards, Sven Klemm

Sven Klemm <sven@timescale.com> writes:

On Mon, Sep 16, 2024 at 5:19 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Configurable to what? If your test cases are dependent on the
historical behavior of PST8PDT, you're out of luck, because that
simply isn't available anymore (or won't be once 2024b reaches
your platform, anyway).

I was wondering whether the timezone used by pg_regress could be made
configurable.

Yes, I understood that you were suggesting that. My point is that
it wouldn't do you any good: you will still have to change any
regression test cases that depend on behavior PST8PDT has/had that
is different from America/Los_Angeles. That being the case,
I don't see much value in making it configurable.

regards, tom lane

Tom Lane:

I was wondering whether the timezone used by pg_regress could be made
configurable.

Yes, I understood that you were suggesting that. My point is that
it wouldn't do you any good: you will still have to change any
regression test cases that depend on behavior PST8PDT has/had that
is different from America/Los_Angeles. That being the case,
I don't see much value in making it configurable.

Just changing it back to PST8PDT wouldn't really help as Tom pointed
out. You'd still get different results depending on which tzdata version
you are running with.

The core regression tests need to be run with a timezone that tests
special cases in the timezone handling code. But that might not be true
for extensions - all they want could be a stable output across major and
minor versions of postgres and versions of tzdata. It could be helpful
to set pg_regress' timezone to UTC, for example?

Best,

Wolfgang

Wolfgang Walther <walther@technowledgy.de> writes:

The core regression tests need to be run with a timezone that tests
special cases in the timezone handling code. But that might not be true
for extensions - all they want could be a stable output across major and
minor versions of postgres and versions of tzdata. It could be helpful
to set pg_regress' timezone to UTC, for example?

I would not recommend that choice. It would mask simple errors such
as failing to apply the correct conversion between timestamptz and
timestamp values. Also, if you have test cases that are affected by
this issue at all, you probably have a need/desire to test things like
DST transitions.

regards, tom lane

On Tue, Sep 17, 2024 at 4:15 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Wolfgang Walther <walther@technowledgy.de> writes:

The core regression tests need to be run with a timezone that tests
special cases in the timezone handling code. But that might not be true
for extensions - all they want could be a stable output across major and
minor versions of postgres and versions of tzdata. It could be helpful
to set pg_regress' timezone to UTC, for example?

I would not recommend that choice. It would mask simple errors such
as failing to apply the correct conversion between timestamptz and
timestamp values. Also, if you have test cases that are affected by
this issue at all, you probably have a need/desire to test things like
DST transitions.

As far as I'm aware timescaledb does not rely on specifics of tzdata
version but just needs a stable setting for timezone. I guess I'll adjust
our tests to not depend on upstream pg_regress timezone.

--
Regards, Sven Klemm