Should I run regression tests?

On Mon, 8 Jun 1998, Doug Lo wrote:

Hi,

I saw the file INSTALL and I ran regression tests, I saw lots of failed tests,
what do they mean?
If I have failed tests, how do I fix it? If I don't run regression tests, what's
the consequences I have?
Would anyone tell me, thanks.

To be honest, the only ppl that should be required to run
regression tests are those that are developing and preparing for
releases...for someone installing, they don't really give a warm fuzzy
feeling due to the discrepencies that the various platforms show that we
consider to be "normal" :(

Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

On Mon, 8 Jun 1998, Doug Lo wrote:

Hi,

I saw the file INSTALL and I ran regression tests, I saw lots of failed tests,
what do they mean?
If I have failed tests, how do I fix it? If I don't run regression tests, what's
the consequences I have?
Would anyone tell me, thanks.

To be honest, the only ppl that should be required to run
regression tests are those that are developing and preparing for
releases...for someone installing, they don't really give a warm fuzzy
feeling due to the discrepencies that the various platforms show that we
consider to be "normal" :(

But INSTALL says:

18) If you wish to skip the regression tests then skip to step 21.
However, we think skipping the tests is a BAD idea!

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

On Mon, 8 Jun 1998, Bruce Momjian wrote:

To be honest, the only ppl that should be required to run
regression tests are those that are developing and preparing for
releases...for someone installing, they don't really give a warm fuzzy
feeling due to the discrepencies that the various platforms show that we
consider to be "normal" :(

But INSTALL says:

18) If you wish to skip the regression tests then skip to step 21.
However, we think skipping the tests is a BAD idea!

and we think this because? its always confused me as to why an
end-user would generally have to run regression tests on "supported and
tested platforms". I can understand us, as developers, doing it prior to
a release, and I can understand someone doing it on an 'untested'
platform...but anything on a supported/tested platform should be caught
by us, the developers, before the end-users see the software...

Now, if we can get the regression tests to pass 100% on all
platforms, the point becomes moot, but, IMHO, all it does is causes/adds
more confusion to the end user then required... :(

Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

Now, if we can get the regression tests to pass 100% on all
platforms, the point becomes moot, but, IMHO, all it does is causes/adds
more confusion to the end user then required... :(

Let's change the INSTALL. We are much more mature now as a product.

-- 
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

At 7:29 +0300 on 8/6/98, The Hermit Hacker wrote:

and we think this because? its always confused me as to why an
end-user would generally have to run regression tests on "supported and
tested platforms". I can understand us, as developers, doing it prior to
a release, and I can understand someone doing it on an 'untested'
platform...but anything on a supported/tested platform should be caught
by us, the developers, before the end-users see the software...

Now, if we can get the regression tests to pass 100% on all
platforms, the point becomes moot, but, IMHO, all it does is causes/adds
more confusion to the end user then required... :(

May I protest, please?

What exactly is a supported/tested platform? Timezone differences make some
of the failures, and I think it's important that we recognise them and know
that we have a timezone problem. Also, have you really tested the system on
all available systems? I saw it compiled for solaris 2.6. Has it been
tested for 2.5? Library differences, a slightly different installation
procedure, and the regression test points you, at least, in the right
direction to ask questions. After all, unix is the administrator's
creation, and he/she may decide to move things around. The regression tests
tell him if one of his inventions are a bit overboard.

End users which merely use the database should not be concerned with such
things, but if we are to run the system in a serious environment, my system
admin wants to be sure that postgres works *here*.

Herouth

--
Herouth Maoz, Internet developer.
Open University of Israel - Telem project
http://telem.openu.ac.il/~herutma

On Mon, 8 Jun 1998, Bruce Momjian wrote:

Now, if we can get the regression tests to pass 100% on all
platforms, the point becomes moot, but, IMHO, all it does is causes/adds
more confusion to the end user then required... :(

Let's change the INSTALL. We are much more mature now as a product.

Agreed, let's just remove the extra line that says that we don't
recommend skipping it...

Moved to pgsql-hackers...

On Mon, 8 Jun 1998, Herouth Maoz wrote:

May I protest, please?

Of course you can...

What exactly is a supported/tested platform?

We have a list of platforms that are tested prior to each
release...

Timezone differences make some
of the failures, and I think it's important that we recognise them and know
that we have a timezone problem. Also, have you really tested the system on
all available systems? I saw it compiled for solaris 2.6. Has it been
tested for 2.5?

Solaris 2.6, 2.5.1 and SunOS 4.1.x were tested for last release by
two ppl (myself included)

End users which merely use the database should not be concerned with such
things, but if we are to run the system in a serious environment, my system
admin wants to be sure that postgres works *here*.

We aren't removing the regression tests, we are just removing the
comment that we strongly encourage ppl to run them...we have yet to have
an end-user report a problem with the regression tests that was something
that was actually a 'bug'...

The Hermit Hacker wrote:

Moved to pgsql-hackers...

On Mon, 8 Jun 1998, Herouth Maoz wrote:

End users which merely use the database should not be concerned with such
things, but if we are to run the system in a serious environment, my system
admin wants to be sure that postgres works *here*.

We aren't removing the regression tests, we are just removing the
comment that we strongly encourage ppl to run them...we have yet to have
an end-user report a problem with the regression tests that was something
that was actually a 'bug'...

Hi,

I'm curious about running regression tests. Why running regression tests is
important
for a ppl not an end-user? If I'm an end-user, running regression tests and get
failed tests,
may I fix'em? If yes, would you like to tell me how to fix? Otherwise, what do
they mean?
Thanks in advance.

Best wishes,
Doug.

The Hermit Hacker wrote:

Moved to pgsql-hackers...

On Mon, 8 Jun 1998, Herouth Maoz wrote:

End users which merely use the database should not be concerned with such
things, but if we are to run the system in a serious environment, my system
admin wants to be sure that postgres works *here*.

We aren't removing the regression tests, we are just removing the
comment that we strongly encourage ppl to run them...we have yet to have
an end-user report a problem with the regression tests that was something
that was actually a 'bug'...

Hi,

I'm curious about running regression tests. Why running regression tests is
important
for a ppl not an end-user? If I'm an end-user, running regression tests and get
failed tests,
may I fix'em? If yes, would you like to tell me how to fix? Otherwise, what do
they mean?
Thanks in advance.

Best wishes,
Doug.

I am glad to hear that Postgresql has gotten so good that the developers
feel that the regression test can be hidden. Hiding it will improve the
appearance of stability. In today's world of mass marketing appearance
seems to count for a lot.

Oliver

The Hermit Hacker wrote:

On Mon, 8 Jun 1998, mark metzger wrote:

I am glad to hear that Postgresql has gotten so good that the developers
feel that the regression test can be hidden. Hiding it will improve the
appearance of stability. In today's world of mass marketing appearance
seems to count for a lot.

Nobody is hiding anything, nor are we removing anything...the
INSTALL guide still lists the regression tests as a step, but it isn't
considered a required step...