Two weeks to feature freeze

We have less than two weeks to feature freeze. Win32 is still in an
uncompleted state, and I haven't been able to return to it recently.
Jan is working on getting exec() working, and hopefully someone can help
me on signals. If I get those two done, I think I can tweek Win32 in
minor ways during beta.

I talked to Patrick about PITR, and with JR now back involved, he might
get it done.

Basically, we might get them both in, or it might be a disaster that we
delayed beta for one month.

What about the nested transaction stuff?

Do we have any "killer" features added to 7.4 that we can shout about?
There's usually been one or two in the past...?

Chris

On Thu, 2003-06-19 at 01:27, Christopher Kings-Lynne wrote:

We have less than two weeks to feature freeze. Win32 is still in an
uncompleted state, and I haven't been able to return to it recently.
Jan is working on getting exec() working, and hopefully someone can help
me on signals. If I get those two done, I think I can tweek Win32 in
minor ways during beta.

I talked to Patrick about PITR, and with JR now back involved, he might
get it done.

Basically, we might get them both in, or it might be a disaster that we
delayed beta for one month.

What about the nested transaction stuff?

Do we have any "killer" features added to 7.4 that we can shout about?
There's usually been one or two in the past...?

A quick glance at the TODO list shows a number of speed improvements in
specific areas (IN, GROUP BY, Subselects in views), ARRAY improvements,
some utility command improvements / additions, and a significant
protocol update.

The protocol update may not be flashy, but it is a large step forward in
presenting a clean experience for developers using PostgreSQL (reduces
chance of rare, unexpected, and difficult to find logic errors).

If nothing else, it makes for an excellent cleanup release that rounds
off some of the sharp corners (tab completion for schema elements in
psql, schema dump in psql, fixed cluster support, transactional
truncate, alter sequence, new regex code for fast MultiByte, etc).

--
Rod Taylor <rbt@rbt.ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

On Wed, 2003-06-18 at 21:27, Christopher Kings-Lynne wrote:

Do we have any "killer" features added to 7.4 that we can shout about?
There's usually been one or two in the past...?

Isn't the index growth problem solved in this release? I think that is
a killer feature that solves a big problem for alot of people.

On Wed, Jun 18, 2003 at 09:59:17PM -0400, Matthew T. O'Connor wrote:

On Wed, 2003-06-18 at 21:27, Christopher Kings-Lynne wrote:

Do we have any "killer" features added to 7.4 that we can shout about?
There's usually been one or two in the past...?

Isn't the index growth problem solved in this release? I think that is
a killer feature that solves a big problem for alot of people.

Yes. That qualifies for "killer features" at least to some big users
here.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Granting software the freedom to evolve guarantees only different results,
not better ones." (Zygo Blaxell)

On Thu, Jun 19, 2003 at 09:27:22AM +0800, Christopher Kings-Lynne wrote:

We have less than two weeks to feature freeze.

What about the nested transaction stuff?

I don't know if it will be completed before feature freeze... we can and
will try, of course. Sadly, like most other people, I have lots of
other things and can't give it the time I wish.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Porque francamente, si para saber manejarse a uno mismo hubiera que
rendir examen... �Qui�n es el machito que tendr�a carnet?" (Mafalda)

"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:

What about the nested transaction stuff?

With all due respect to Alvaro et al, I can't imagine that that will
make it into 7.4. (I have no confidence that PITR or Win32 native port
will make it either...)

Do we have any "killer" features added to 7.4 that we can shout about?

We have a lot of pretty good stuff. You're not happy that the
performance of IN (subselect) has been fixed? That btree index bloat is
fixed (at least in large part, it remains to be seen whether the field
performance is all that we need...)?

In my opinion the project is not at a state where whizzy new Features
with a capital F are going to jump out of the woodwork. We are making
good advances in performance, reliability, SQL spec compliance, and
stuff like that, but fancy-sounding bullet points are hard to come by.

I can tell you that Red Hat's CCM group (the former Ars Digita) is
waiting with bated breath for 7.4, because it fixes a number of problems
(IN-subselect being one) that prevent 7.3 from being a serious
competitor to Oracle for their platform. 7.4 is a killer release for
them, and has been since about February, and they're getting tired of
waiting. I think a lot of other people are in the same situation,
even though they may not know it ;-)

We can't slip this puppy any more --- it's time to wrap her up and
push her out.

regards, tom lane

On Thu, 19 Jun 2003, Christopher Kings-Lynne wrote:

We have less than two weeks to feature freeze. Win32 is still in an
uncompleted state, and I haven't been able to return to it recently.
Jan is working on getting exec() working, and hopefully someone can help
me on signals. If I get those two done, I think I can tweek Win32 in
minor ways during beta.

I talked to Patrick about PITR, and with JR now back involved, he might
get it done.

Basically, we might get them both in, or it might be a disaster that we
delayed beta for one month.

What about the nested transaction stuff?

Do we have any "killer" features added to 7.4 that we can shout about?
There's usually been one or two in the past...?

I'm not sure if contrib/tsearch is a "killer" feature, but we hope
to submit completely new version of tsearch V2 before July 1.
Actually, we have stable code already used in some projects but
currently lacking documentation. Several people are working on tutorial,
reference guide. The problem is that Bruce seems is very overloaded and
for sure he'll have many patches close to July 1. Is it possible
to get rights to commit our changes ?

Chris

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

Maybe a better strategy would be to get a release out soon but not wait 6
months for another release which would contain the Win32 port and the PITR
stuff (assuming those aren't done in time for this release).

Just a thought.

andrew

Tom Lane wrote:

On Thursday 19 June 2003 03:27, Christopher Kings-Lynne wrote:

Do we have any "killer" features added to 7.4 that we can shout about?

We should not forget the availability of PostgreSQL companion products, like
pgAdmin3 and PhpPgAdmin3. These two GUIs should be ready for release during
July, although I am not in the shoes of the project leaders, Dave and
Christopher and cannot speak of it officially.

They are not part of the PostgreSQL distribution, but well could be.

Providing a reliable bundle including PostgreSQL, a graphical GUI (pgAdmin3)
and a web administration interface (PhpPgAdmin3) is a big news. This will
convince entry users, who normally turn to MySQL, that PostgreSQL is the best
choice. We are not downgrading features but upgrading user needs...

When PostgreSQL win32 port is ready, all platforms and entry user needs will
be covered. Isn't it a big news? Just my 2 cents...

Best regards,
Jean-Michel POURE

On Wed, 2003-06-18 at 23:07, Tom Lane wrote:

"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:

What about the nested transaction stuff?

With all due respect to Alvaro et al, I can't imagine that that will
make it into 7.4. (I have no confidence that PITR or Win32 native port
will make it either...)

Heres hoping for win32, that is a killer feature for so many people and
we're so close to it...

Do we have any "killer" features added to 7.4 that we can shout about?

We have a lot of pretty good stuff. You're not happy that the
performance of IN (subselect) has been fixed? That btree index bloat is
fixed (at least in large part, it remains to be seen whether the field
performance is all that we need...)?

I think the auto vacuum work will be pretty big, and I personally think
statement level triggers are pretty important too. (Which reminds me I
really need to start banging on those a bit more.)

In my opinion the project is not at a state where whizzy new Features
with a capital F are going to jump out of the woodwork. We are making
good advances in performance, reliability, SQL spec compliance, and
stuff like that, but fancy-sounding bullet points are hard to come by.

You mean like that other database that just recently added transaction
support ;-)

I do see a number of capital F features that haven't been done yet,
win32, replication, nested transactions... imho those features could
each warrant a development cycle on their own.

I can tell you that Red Hat's CCM group (the former Ars Digita) is
waiting with bated breath for 7.4, because it fixes a number of problems
(IN-subselect being one) that prevent 7.3 from being a serious
competitor to Oracle for their platform. 7.4 is a killer release for
them, and has been since about February, and they're getting tired of
waiting. I think a lot of other people are in the same situation,
even though they may not know it ;-)

We can't slip this puppy any more --- it's time to wrap her up and
push her out.

Well, I suppose that history has shown that waiting on specific features
causes trouble with postgresql development, but I don't see why a
release can't be based around waiting for feature x as long as feature x
is being actively worked on by trusted developers who have an endgame in
sight.

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat <xzilla@users.sourceforge.net> writes:

Well, I suppose that history has shown that waiting on specific features
causes trouble with postgresql development, but I don't see why a
release can't be based around waiting for feature x as long as feature x
is being actively worked on by trusted developers who have an endgame in
sight.

We have been led down that garden path before, and it's been a losing
proposition every time.

regards, tom lane

On Thu, 2003-06-19 at 06:12, Andrew Dunstan wrote:

Maybe a better strategy would be to get a release out soon but not wait 6
months for another release which would contain the Win32 port and the PITR
stuff (assuming those aren't done in time for this release).

Thats what Justin was saying about this one, that it should be released
early due to win32 being complete.

I would expect, even once it compiles and runs (it doesn't yet) that it
will need some good testing before being released in any form. Platform
changes of this significance tend to introduce lots of new and
unexpected issues.

Tracking down several users for windows testing is a good idea. Rushing
it out in an unknown state to get the testing isn't such a good idea in
my mind.

--
Rod Taylor <rbt@rbt.ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

Tom wrote:

Do we have any "killer" features added to 7.4 that we can shout about?

We have a lot of pretty good stuff. You're not happy that the
performance of IN (subselect) has been fixed? That btree index bloat is
fixed...

For warehousing & reporting, "Add hash for evaluating GROUP BY aggregates"
can be a killer feature.

7.4 is a killer release for them, and has been since about February,
and they're getting tired of waiting. I think a lot of other people
are in the same situation, even though they may not know it ;-)

A nightly reporting process that starts here at midnight each night
takes about 12 hours on 7.3 and about 9 hours on 7.4alpha; possibly
thanks to HashAggregates. While this may not sound like much, it
means Marketing could see results when they arive at work, instead
of waiting for afternoon.

The perceived improvement of "ready before work" vs. "wait three hours"
is a killer feature for this system.

Ron

On Thu, 19 Jun 2003, Robert Treat wrote:

Well, I suppose that history has shown that waiting on specific features
causes trouble with postgresql development, but I don't see why a
release can't be based around waiting for feature x as long as feature x
is being actively worked on by trusted developers who have an endgame in
sight.

Everyone has an 'endgame in sight', at least when they ask for a release
to be postponed ... but then their date keeps slipping, etc ...

The thing is, if win32 is 'that close to being finished', then as soon as
v7.4 is out, that code should be ready to throw in ... and the same for
every other features that could 'postpone a release' ...

I'd rather see the dev cycle shortened by a month, then extended ...

We have a lot of pretty good stuff. You're not happy that the
performance of IN (subselect) has been fixed?

All our code uses workaround now :)

That btree index bloat is
fixed (at least in large part, it remains to be seen whether the field
performance is all that we need...)?

Yes, that's a good feature.

We can't slip this puppy any more --- it's time to wrap her up and
push her out.

OK, let's do it - we've got full support for it in phpPgAdmin already :)

Chris

On Thu, 19 Jun 2003, Andrew Dunstan wrote:

Maybe a better strategy would be to get a release out soon but not wait
6 months for another release which would contain the Win32 port and the
PITR stuff (assuming those aren't done in time for this release).

Just a thought.

And definitely in agreement here ... I'd rather see a shortened dev cycle
prompted by a big feature being added, then delaying a release because "oh
oh, I need another few weeks" that draws out when something unexpected
happens :(

andrew

Tom Lane wrote:

Robert Treat <xzilla@users.sourceforge.net> writes:

Well, I suppose that history has shown that waiting on specific
features causes trouble with postgresql development, but I don't see
why a release can't be based around waiting for feature x as long as
feature x is being actively worked on by trusted developers who have
an endgame in sight.

We have been led down that garden path before, and it's been a losing
proposition every time.

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

On Thu, 19 Jun 2003, Oleg Bartunov wrote:

I'm not sure if contrib/tsearch is a "killer" feature, but we hope to
submit completely new version of tsearch V2 before July 1. Actually, we
have stable code already used in some projects but currently lacking
documentation. Several people are working on tutorial, reference guide.
The problem is that Bruce seems is very overloaded and for sure he'll
have many patches close to July 1. Is it possible to get rights to
commit our changes ?

Is there a strong reason why tsearch isn't in gborg?

On Thu, 19 Jun 2003, The Hermit Hacker wrote:

On Thu, 19 Jun 2003, Oleg Bartunov wrote:

I'm not sure if contrib/tsearch is a "killer" feature, but we hope to
submit completely new version of tsearch V2 before July 1. Actually, we
have stable code already used in some projects but currently lacking
documentation. Several people are working on tutorial, reference guide.
The problem is that Bruce seems is very overloaded and for sure he'll
have many patches close to July 1. Is it possible to get rights to
commit our changes ?

Is there a strong reason why tsearch isn't in gborg?

How gborg could help us submitting changes to pgsql CVS ?

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

On Fri, 20 Jun 2003, Oleg Bartunov wrote:

Is there a strong reason why tsearch isn't in gborg?

How gborg could help us submitting changes to pgsql CVS ?

It wouldn't ... is there a reason why tsearch needs to be in the pgsql CVS
any more then, say, ODBC drivers, or the tcl interface, or the python
interface, or ... ?

On Fri, 20 Jun 2003, The Hermit Hacker wrote:

On Fri, 20 Jun 2003, Oleg Bartunov wrote:

Is there a strong reason why tsearch isn't in gborg?

How gborg could help us submitting changes to pgsql CVS ?

It wouldn't ... is there a reason why tsearch needs to be in the pgsql CVS
any more then, say, ODBC drivers, or the tcl interface, or the python
interface, or ... ?

because tsearch lives under pgsql source tree, in contrib directory though.
I'd never asked for any rights (I have enough beside postgresq :),
but practice of development life is so, that Bruce has no time to apply
them in reasonable time. Last patch waited about 1 month.
I'd like to
see tsearch in contrib directory because we have direct benefit from that -
rather often Tom change contrib sources to sync with his changes in
main tree. This is the only real reason why I want to stay under postgres
tree. I was afraid, and Bruce actually wrote he will be busy,
we couldn't submit our new version.

Freebsd has separate rights for core and ports.

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

On Fri, 20 Jun 2003, The Hermit Hacker wrote:
Is there a strong reason why tsearch isn't in gborg?

I think text search is a pretty important facility that should
eventually be part of the core distribution. It's more likely to get
there from contrib than from gborg ...

regards, tom lane

On Fri, 20 Jun 2003, Tom Lane wrote:

On Fri, 20 Jun 2003, The Hermit Hacker wrote:
Is there a strong reason why tsearch isn't in gborg?

I think text search is a pretty important facility that should
eventually be part of the core distribution. It's more likely to get
there from contrib than from gborg ...

Why part of the core distribution, and not just left as a loadable module,
like it is now?

Why part of the core distribution, and not just left as a loadable module,
like it is now?

The day I can go 'CREATE FULLTEXT INDEX ...' just like MySQL can I will be a
very happy chappy...

Chris

The Hermit Hacker kirjutas R, 20.06.2003 kell 08:28:

On Fri, 20 Jun 2003, Tom Lane wrote:

On Fri, 20 Jun 2003, The Hermit Hacker wrote:
Is there a strong reason why tsearch isn't in gborg?

I think text search is a pretty important facility that should
eventually be part of the core distribution. It's more likely to get
there from contrib than from gborg ...

Why part of the core distribution, and not just left as a loadable module,
like it is now?

I remember Tom saying that builtin functions calls are a lot faster than
loadable C functions.

If that can be fixed, then it *could* stay loadable.

Also, having built-in full text indexing is very desirable. And I don't
see any even nearly as good competing fulltext indexing modules
anywhere.

If we had to move something *out* of core in order to get tsearch in,
then I personally would not mind if all geometry types go to gborg, but
I'm sure there are some users who would mind ;)

---------------
Hannu

On Thu, 19 Jun 2003, The Hermit Hacker wrote:

On Thu, 19 Jun 2003, Andrew Dunstan wrote:

Maybe a better strategy would be to get a release out soon but not wait
6 months for another release which would contain the Win32 port and the
PITR stuff (assuming those aren't done in time for this release).

Just a thought.

And definitely in agreement here ... I'd rather see a shortened dev cycle
prompted by a big feature being added, then delaying a release because "oh
oh, I need another few weeks" that draws out when something unexpected
happens :(

...

I'm not sure why another delay is being considered. There's been a delay of
a week because of the server problems hasn't there and wasn't the original
delay only acceptable on the basis that that was that and there wasn't going to
be another extension?

--
Nigel Andrews

On Fri, 20 Jun 2003, Christopher Kings-Lynne wrote:

Why part of the core distribution, and not just left as a loadable module,
like it is now?

The day I can go 'CREATE FULLTEXT INDEX ...' just like MySQL can I will be a
very happy chappy...

with new tserach v2 we're pretty close to that day. we need more testing,
more suggestions and more documentation. There are several issues remains,
mostly with core GiST not tsearch. The most important is concurrency support !
We've several times planned to work on it, but real life is rather complex
thing :(

Chris

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

The Hermit Hacker wrote:

On Thu, 19 Jun 2003, Andrew Dunstan wrote:

Maybe a better strategy would be to get a release out soon but not wait
6 months for another release which would contain the Win32 port and the
PITR stuff (assuming those aren't done in time for this release).

Just a thought.

And definitely in agreement here ... I'd rather see a shortened dev cycle
prompted by a big feature being added, then delaying a release because "oh
oh, I need another few weeks" that draws out when something unexpected
happens :(

Yep, this makes sense. Looks like it'll be PostgreSQL 7.4 being all the
present improvements, but without PITR and Win32. Then, in a few months
(hopefully less than 3) we'll have PostgreSQL 8.0, with both of those
major features in it (and whatever other enhancements have been added).

The only thing that makes me wince is that we have a protocol change at
PostgreSQL 7.4 release instead of 8.0. It kind of doesn't sound right,
having a protocol change in the "7 series", when we have an "8 series"
coming up soon after.

Oh well, so it's not perfect...

;-)

Regards and best wishes,

Justin Clift

<snip>

On Fri, 2003-06-20 at 04:41, Nigel J. Andrews wrote:

On Thu, 19 Jun 2003, The Hermit Hacker wrote:

On Thu, 19 Jun 2003, Andrew Dunstan wrote:

Maybe a better strategy would be to get a release out soon but not wait
6 months for another release which would contain the Win32 port and the
PITR stuff (assuming those aren't done in time for this release).

Just a thought.

And definitely in agreement here ... I'd rather see a shortened dev cycle
prompted by a big feature being added, then delaying a release because "oh
oh, I need another few weeks" that draws out when something unexpected
happens :(

...

I'm not sure why another delay is being considered. There's been a delay of
a week because of the server problems hasn't there and wasn't the original
delay only acceptable on the basis that that was that and there wasn't going to
be another extension?

There really isn't for this release. Any talk of delay is just a thought
on general policy for future releases.

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

On Fri, 2003-06-20 at 06:59, Justin Clift wrote:

The Hermit Hacker wrote:

On Thu, 19 Jun 2003, Andrew Dunstan wrote:

Yep, this makes sense. Looks like it'll be PostgreSQL 7.4 being all the
present improvements, but without PITR and Win32. Then, in a few months
(hopefully less than 3) we'll have PostgreSQL 8.0, with both of those
major features in it (and whatever other enhancements have been added).

The only thing that makes me wince is that we have a protocol change at
PostgreSQL 7.4 release instead of 8.0. It kind of doesn't sound right,
having a protocol change in the "7 series", when we have an "8 series"
coming up soon after.

Oh well, so it's not perfect...

...which is why I'd advocate making this release an 8.0 regardless of
win32 or pitr. I know it's old school to actually base versioning on
technical criteria instead of buzzwords, but there's no reason we have
to follow the corporate mold. Still, I'd rather not get into that debate
yet since I don't want to let the win32 guys off the hook yet!

win32 for the next release! go guys go!

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat <xzilla@users.sourceforge.net> writes:

On Fri, 2003-06-20 at 06:59, Justin Clift wrote:

The only thing that makes me wince is that we have a protocol change at
PostgreSQL 7.4 release instead of 8.0.

...which is why I'd advocate making this release an 8.0 regardless of
win32 or pitr.

<shrug> ... The backend will still talk to old clients, and libpq will
still talk to old backends, so I don't think the protocol change is
really going to cause a flag day for anyone. On a technical level it's
probably not an adequate reason to call this release 8.0.

On the other hand, I dislike the notion that we would call a release 8.0
simply because it now has a native Windows port. (And if there is a
short release cycle after this one, that might be about the only big new
thing there.) Considering that we aren't going to be recommending the
Windows port for production work, we should not let the release
numbering give the impression we think it's the greatest Postgres
feature ever to come down the pike.

I'm happy to keep calling 'em 7.* for the foreseeable future, myself.

regards, tom lane

And, actually, for some reason I hadn't thought of the tsearch as being
another 'INDEX' type ... I crawl back over and be quiet now :)

Oleg, as far as commits are concerned, I have no problems with extending
the privileges to one of your guys for this, just email me seperately who,
and I'll get it setup ...

On Fri, 20 Jun 2003, Oleg Bartunov wrote:

On Fri, 20 Jun 2003, Christopher Kings-Lynne wrote:

Why part of the core distribution, and not just left as a loadable module,
like it is now?

The day I can go 'CREATE FULLTEXT INDEX ...' just like MySQL can I will be a
very happy chappy...

with new tserach v2 we're pretty close to that day. we need more testing,
more suggestions and more documentation. There are several issues remains,
mostly with core GiST not tsearch. The most important is concurrency support !
We've several times planned to work on it, but real life is rather complex
thing :(

Chris

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

On Fri, 2003-06-20 at 10:42, Tom Lane wrote:

Robert Treat <xzilla@users.sourceforge.net> writes:

On Fri, 2003-06-20 at 06:59, Justin Clift wrote:

The only thing that makes me wince is that we have a protocol change at
PostgreSQL 7.4 release instead of 8.0.

...which is why I'd advocate making this release an 8.0 regardless of
win32 or pitr.

<shrug> ... The backend will still talk to old clients, and libpq will
still talk to old backends, so I don't think the protocol change is
really going to cause a flag day for anyone. On a technical level it's
probably not an adequate reason to call this release 8.0.

Can you give me an example of a technical change that would warrant a
major version bump?

On the other hand, I dislike the notion that we would call a release 8.0
simply because it now has a native Windows port. (And if there is a
short release cycle after this one, that might be about the only big new
thing there.) Considering that we aren't going to be recommending the
Windows port for production work, we should not let the release
numbering give the impression we think it's the greatest Postgres
feature ever to come down the pike.

yep.

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert,

Well, I suppose that history has shown that waiting on specific features
causes trouble with postgresql development, but I don't see why a
release can't be based around waiting for feature x as long as feature x
is being actively worked on by trusted developers who have an endgame in
sight.

Ultimately, this is one of those "technical" vs. "marketing" questions ...
whether to release now with a bunch of back-end features that the current
users want, or to release later and include the features that we said were
going to be in 7.4. And PostgreSQL is a technical project, not a marketing
one.

I know that, given MySQL's attempts to squeeze PostgreSQL out of the market,
that there is a lot of desire to include at least one "killer feature" in
each release to grab press coverage. But the PostgreSQL project can't let
our technical decisions be governed by our PR, or we *become* MySQL.

--
Josh Berkus
Aglio Database Solutions
San Francisco

Robert Treat <xzilla@users.sourceforge.net> writes:

On Fri, 2003-06-20 at 10:42, Tom Lane wrote:

<shrug> ... The backend will still talk to old clients, and libpq will
still talk to old backends, so I don't think the protocol change is
really going to cause a flag day for anyone. On a technical level it's
probably not an adequate reason to call this release 8.0.

Can you give me an example of a technical change that would warrant a
major version bump?

Well, if we hadn't gotten the work done to make libpq still able to talk
to older backends, then we'd have had enough of a compatibility issue
that I think calling it 8.0 would have been a reasonable thing to do.

If you want a feature-with-a-capital-F reason for going to 8.0, there is
only one candidate Feature in my personal view, and that's a built-in
replication solution. That doesn't seem to be getting any nearer :-(

regards, tom lane

On Fri, 2003-06-20 at 11:21, Josh Berkus wrote:

Robert,

Well, I suppose that history has shown that waiting on specific features
causes trouble with postgresql development, but I don't see why a
release can't be based around waiting for feature x as long as feature x
is being actively worked on by trusted developers who have an endgame in
sight.

Ultimately, this is one of those "technical" vs. "marketing" questions ...

absolutely not. this is a x style of development vs. y style of
development discussion. many many projects, commercial and open source,
use a style of releasing based on features included in a given version.
In fact I'd be willing to say that the majority of open source projects
work this way, since open source projects generally aren't beholden to
release dates, giving developers the time they need to get specific
features done and release them when they are ready. as i prefaced in my
message, "history has shown us that waiting on specific features causes
trouble with postgresql development", but that doesn't mean we should
act as if this style of development doesn't exist.

whether to release now with a bunch of back-end features that the current
users want, or to release later and include the features that we said were
going to be in 7.4. And PostgreSQL is a technical project, not a marketing
one.

right, which is why core/hackers will decide both what gets into each
releases and when it's released. since i'm not outpacing tom or bruce
in my patch submissions i don't expect them to bend to my whims, but as
someone who follows and participates in postgresql development
regardless of an marketing aspects, i don't mind pointing out
alternatives.

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert,

absolutely not. this is a x style of development vs. y style of
development discussion. many many projects, commercial and open source,
use a style of releasing based on features included in a given version.
In fact I'd be willing to say that the majority of open source projects
work this way, since open source projects generally aren't beholden to
release dates, giving developers the time they need to get specific
features done and release them when they are ready. as i prefaced in my
message, "history has shown us that waiting on specific features causes
trouble with postgresql development", but that doesn't mean we should
act as if this style of development doesn't exist.

Ah. I see what you mean. Sorry for the misunderstanding.

The thing is, from the perspective of *current* Postgres users, 7.4 has
several "killer" features, some of which have been ready for 3 months. In
fact, I just finished sending an e-mail to a client advising them to try 7.4
as soon as it is tested, becuase of hashaggs.

So looked at from that perspective, our mistake was to try to cram too many
features into 7.4 ... more than could possibly get done in 6-8 months.
What's happening now is that the core group has decided, OK, we don't have
5-6 of the features we wanted to have, but we do have 10 other features, so
maybe it's time to release.

I'm not sure you're correct in the behaviour of the majoirty of OSS projects.
Certainly if the Mozilla or OpenOffice.org projects are attaching specific
release numbers to specific features, I haven't seen it. Linux does that, I
guess, but that does result in a 2-year span between major releases -- and
results in a lot of major features being included in "incremental" releases.
But I think most OSS projects just release when they think they have enough
tested features to justify a major release -- regardless of what those
features are.

Anybody here on other projects want to weigh in?

Me, I'd be in favor of releasing more frequently ... I felt that we waited too
long for 7.4.

--
Josh Berkus
Aglio Database Solutions
San Francisco

Josh Berkus <josh@agliodbs.com> writes:

So looked at from that perspective, our mistake was to try to cram too many
features into 7.4 ... more than could possibly get done in 6-8 months.

It's more that we thought that all these features would get done in
about the same timeframe, and (not too surprisingly) some got done and
some didn't.

Me, I'd be in favor of releasing more frequently ... I felt that we
waited too long for 7.4.

Yeah, this is why I'm not in favor of slipping more.

Time was that we had a major release every 3 or 4 months. As the project
matures I think it's appropriate for the cycle to get slower: a lot of
low-hanging fruit is gone, so we have larger jobs to tackle, plus users
are using PG for larger databases and don't want to face major-version
changes too often. But I don't want it to get to be a year on average
between releases, at least not yet. 8 or 9 months seems reasonable, and
by that standard we're overdue.

regards, tom lane

The Hermit Hacker <scrappy@postgresql.org> writes:

On Thu, 19 Jun 2003, Robert Treat wrote:

Well, I suppose that history has shown that waiting on specific features
causes trouble with postgresql development, but I don't see why a
release can't be based around waiting for feature x as long as feature x
is being actively worked on by trusted developers who have an endgame in
sight.

Everyone has an 'endgame in sight', at least when they ask for a
release to be postponed ... but then their date keeps slipping, etc
...

The thing is, if win32 is 'that close to being finished', then as
soon as v7.4 is out, that code should be ready to throw in ... and
the same for every other features that could 'postpone a release'
...

I'd rather see the dev cycle shortened by a month, then extended ...

Why couldn't you just release the win32 version of 7.4 when it was
finished. If it takes an extra month then that just gives you guys
the chance to circulate *two* press releases. The Native Win32 port
is likely to make a big enough splash all by itself.

Jason

-----Original Message-----
From: Jason Earl [mailto:jason.earl@simplot.com]
Sent: Friday, June 20, 2003 10:45 AM
To: The Hermit Hacker
Cc: Robert Treat; Tom Lane; Christopher Kings-Lynne; Bruce
Momjian; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

The Hermit Hacker <scrappy@postgresql.org> writes:

On Thu, 19 Jun 2003, Robert Treat wrote:

Well, I suppose that history has shown that waiting on specific
features causes trouble with postgresql development, but I

don't see

why a release can't be based around waiting for feature x

as long as

feature x is being actively worked on by trusted

developers who have

an endgame in sight.

Everyone has an 'endgame in sight', at least when they ask for a
release to be postponed ... but then their date keeps slipping, etc
...

The thing is, if win32 is 'that close to being finished',

then as soon

as v7.4 is out, that code should be ready to throw in ...

and the same

for every other features that could 'postpone a release' ...

I'd rather see the dev cycle shortened by a month, then extended ...

Why couldn't you just release the win32 version of 7.4 when
it was finished. If it takes an extra month then that just
gives you guys the chance to circulate *two* press releases.
The Native Win32 port is likely to make a big enough splash
all by itself.

A formal release needs a big testing effort. Two separate releases will
double the work of validation.

Dann Corbit wrote:

-----Original Message-----
From: Jason Earl [mailto:jason.earl@simplot.com]
Sent: Friday, June 20, 2003 10:45 AM
To: The Hermit Hacker
Cc: Robert Treat; Tom Lane; Christopher Kings-Lynne; Bruce
Momjian; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

[...]

Why couldn't you just release the win32 version of 7.4 when
it was finished. If it takes an extra month then that just
gives you guys the chance to circulate *two* press releases.
The Native Win32 port is likely to make a big enough splash
all by itself.

A formal release needs a big testing effort. Two separate releases will
double the work of validation.

That's true in the general case.

But in this case we're talking about releasing for a completely new
platform. That's much different than doing another release for the same
platform set.

The testing that needs to be done for the Win32 release has to be done
separately *anyway*, so there's nothing lost by releasing the Win32 port
separately.

--
Kevin Brown kevin@sysexperts.com

"Dann Corbit" <DCorbit@connx.com> writes:

Why couldn't you just release the win32 version of 7.4 when
it was finished. If it takes an extra month then that just
gives you guys the chance to circulate *two* press releases.
The Native Win32 port is likely to make a big enough splash
all by itself.

A formal release needs a big testing effort. Two separate releases
will double the work of validation.

There are several problems with that statement. The first is that
PostgreSQL's "testing effort" happens right here on this mailing list.
The various PostgreSQL hackers code stuff up, and we try and break it.
There's very little /effort/ involved. People that want the new
features go out on a limb and start using them. If they don't work,
then they bring it up on the list. If they do work then very little
gets said.

As it now stands Tom Lane is on the record as stating that the new
Win32 version isn't going to be ready for production anyhow. If
anything the Win32 version *should* get released separately simply
because we don't want people mistaking the Win32 version as being up
to the PostgreSQL teams high standards. Those people that want the
Win32 version to become production ready are going to have to risk
their precious data. Otherwise, the Win32 version will likely remain
a second class citizen forever.

The fact of the matter is that the Win32 specific bits are the parts
that are likely to break in the new port. If anything the Windows
version will *benefit* from an earlier *nix release because the *nix
users will chase down the bugs in the new PostgreSQL features. Once
the *nix version is up to 7.4.2 (or so) then a Windows release of
7.4.2 should allow the PostgreSQL hackers to simply chase down Windows
specific problems.

Adding a new platform--especially a platform as diverse from the rest
of PostgreSQL's supported platforms as Windows--is what adds the work.
Testing the new platform is relatively easy. All you need to do is to
start using the Win32 version with real live data.

Jason

-----Original Message-----
From: Jason Earl [mailto:jason.earl@simplot.com]
Sent: Friday, June 20, 2003 3:32 PM
To: Dann Corbit
Cc: Jason Earl; The Hermit Hacker; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

"Dann Corbit" <DCorbit@connx.com> writes:

Why couldn't you just release the win32 version of 7.4 when
it was finished. If it takes an extra month then that just
gives you guys the chance to circulate *two* press releases.
The Native Win32 port is likely to make a big enough splash
all by itself.

A formal release needs a big testing effort. Two separate releases
will double the work of validation.

There are several problems with that statement. The first is
that PostgreSQL's "testing effort" happens right here on this
mailing list.

That's not exactly reassuring. There is no regression test that gets
formal acceptance?!

The various PostgreSQL hackers code stuff up,
and we try and break it. There's very little /effort/
involved. People that want the new features go out on a limb
and start using them. If they don't work, then they bring it
up on the list. If they do work then very little gets said.

As it now stands Tom Lane is on the record as stating that
the new Win32 version isn't going to be ready for production
anyhow. If anything the Win32 version *should* get released
separately simply because we don't want people mistaking the
Win32 version as being up to the PostgreSQL teams high
standards. Those people that want the Win32 version to
become production ready are going to have to risk their
precious data. Otherwise, the Win32 version will likely
remain a second class citizen forever.

The fact of the matter is that the Win32 specific bits are
the parts that are likely to break in the new port. If
anything the Windows version will *benefit* from an earlier
*nix release because the *nix users will chase down the bugs
in the new PostgreSQL features. Once the *nix version is up
to 7.4.2 (or so) then a Windows release of 7.4.2 should allow
the PostgreSQL hackers to simply chase down Windows specific problems.

Then using the same logic, the new Windows version should wait
indefinitely, since the *nix version will always be shaking out bugs.

Adding a new platform--especially a platform as diverse from
the rest of PostgreSQL's supported platforms as Windows--is
what adds the work. Testing the new platform is relatively
easy. All you need to do is to start using the Win32 version
with real live data.

That is not testing. Using the world as your beta team seems to be a
business model used by a few software giants that is largely frowned
upon. I would think that there is an opportunity to do things
differently. [Read 'properly'].

We (at CONNX Solutions Inc.) have a formal release procedure that
includes many tens of thousands of automated tests using dozens of
different platforms. There are literally dozens of machines (I would
guess 70 or so total) running around the clock for 7 days before we even
know if we have a release candidate. The QA team is distinct from the
development team, and if they say "FLOP!" the release goes nowhere. No
formal release until QA passes it.

If there is no procedure for PostgreSQL of this nature, then there
really needs to be. I am sure that MySQL must have something in place
like that. Their "Crash-Me" test suite shows (at least) that they have
put a large effort into testing.

On Fri, Jun 20, 2003 at 03:39:47PM -0700, Dann Corbit wrote:

We (at CONNX Solutions Inc.) have a formal release procedure that
includes many tens of thousands of automated tests using dozens of
different platforms. [...]

If there is no procedure for PostgreSQL of this nature, then there
really needs to be. I am sure that MySQL must have something in place
like that. Their "Crash-Me" test suite shows (at least) that they have
put a large effort into testing.

The regression testing suite in Postgres is one of the things that
impresses me about this software. It's very rare that a change is even
committed to the main tree if a single regression test doesn't pass.
When it does, a proper fix is quickly put in or the change reverted.

It's even rare that patches with regression failures get posted in
pgsql-patches. Regression tests are a very handy tool for contributors
to check that their work is "safe". It's considered good practice to
submit new tests when there's new functionality in a patch.

There probably isn't such a gigantic effort like the one you describe,
but there certainly _is_ a testing procedure. There's probably room for
improvement, of course, but we don't want the tests to take a full week
to complete, IMHO.

It would be nice to have a system which could receive a patch and
compile and verify that it passes the tests before it goes to Bruce's
queue; or compile on multiple platforms to check for portability
problems, for example.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Uno puede defenderse de los ataques; contra los elogios se esta indefenso"

"Dann Corbit" <DCorbit@connx.com> writes:

-----Original Message-----
From: Jason Earl [mailto:jason.earl@simplot.com]
Sent: Friday, June 20, 2003 3:32 PM
To: Dann Corbit
Cc: Jason Earl; The Hermit Hacker; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

"Dann Corbit" <DCorbit@connx.com> writes:

Why couldn't you just release the win32 version of 7.4 when
it was finished. If it takes an extra month then that just
gives you guys the chance to circulate *two* press releases.
The Native Win32 port is likely to make a big enough splash
all by itself.

A formal release needs a big testing effort. Two separate releases
will double the work of validation.

There are several problems with that statement. The first is
that PostgreSQL's "testing effort" happens right here on this
mailing list.

That's not exactly reassuring. There is no regression test that
gets formal acceptance?!

Yes, there are regression tests, and new tests get invented all of the
time whenever the real world finds new bugs. Regression tests are
excellent for making sure that you don't make the same mistake twice,
but they aren't a substitute for handing the code over to actual end
users.

The various PostgreSQL hackers code stuff up,
and we try and break it. There's very little /effort/
involved. People that want the new features go out on a limb
and start using them. If they don't work, then they bring it
up on the list. If they do work then very little gets said.

As it now stands Tom Lane is on the record as stating that
the new Win32 version isn't going to be ready for production
anyhow. If anything the Win32 version *should* get released
separately simply because we don't want people mistaking the
Win32 version as being up to the PostgreSQL teams high
standards. Those people that want the Win32 version to
become production ready are going to have to risk their
precious data. Otherwise, the Win32 version will likely
remain a second class citizen forever.

The fact of the matter is that the Win32 specific bits are
the parts that are likely to break in the new port. If
anything the Windows version will *benefit* from an earlier
*nix release because the *nix users will chase down the bugs
in the new PostgreSQL features. Once the *nix version is up
to 7.4.2 (or so) then a Windows release of 7.4.2 should allow
the PostgreSQL hackers to simply chase down Windows specific problems.

Then using the same logic, the new Windows version should wait
indefinitely, since the *nix version will always be shaking out
bugs.

That's not true at all. Despite the excellent work by the PostgreSQL
team, and despite the beta testing that will be done by volunteers, if
history repeats itself, there will be problems with version 7.4.0,
even on platforms that have been well supported by PostgreSQL forever.
I am not saying that we should hold off indefinitely on the Win32
port, I am simply saying that it probably wouldn't hurt to shake out
the normal .0 release bugs before throwing the unique Win32 bugs into
the mix.

My guess is that reported Win32 bugs are going blamed on the Win32
specific bits at first no matter what happens. Unless the bug can be
demonstrated on a *nix version it will be assumed that the problem is
a shortcoming of the Win32 specific code. That's just common sense.

Adding a new platform--especially a platform as diverse from
the rest of PostgreSQL's supported platforms as Windows--is
what adds the work. Testing the new platform is relatively
easy. All you need to do is to start using the Win32 version
with real live data.

That is not testing. Using the world as your beta team seems to be
a business model used by a few software giants that is largely
frowned upon. I would think that there is an opportunity to do
things differently. [Read 'properly'].

Hmm... I must have missed the huge corporation paying for in house
testing of PostgreSQL. In the Free Software world the "beta team" is
all of those people that need the new features so badly that they are
willing to risk their own data and hardware testing it. You might not
like the way that this sounds, but in practice it works astoundingly
well. Chances are you can't name 25 pieces of commercial software
that run on the wide array of hardware platforms and OSes as
PostgreSQL, and PostgreSQL has a earned a well deserved reputation for
being a solid piece of software. Clearly the PostgreSQL team is doing
*something* right.

We (at CONNX Solutions Inc.) have a formal release procedure that
includes many tens of thousands of automated tests using dozens of
different platforms. There are literally dozens of machines (I
would guess 70 or so total) running around the clock for 7 days
before we even know if we have a release candidate. The QA team is
distinct from the development team, and if they say "FLOP!" the
release goes nowhere. No formal release until QA passes it.

And yet when you release the software your customers invariably find
bugs, don't they?

Don't get me wrong. I am all for testing, regression tests, and such,
but the fact of the matter is that there simply is no way that a
centralized authority could afford to really test PostgreSQL on even a
fraction of the supported platforms and configurations. The way it
stands now the PostgreSQL teams gets the best testbed you could hope
for (the world) for the price of hosting a few web and FTP servers
(thanks Marc).

PostgreSQL betas almost certainly gest tested on an order of magnitude
more systems than the 70 that you boast about. PostgreSQL gets tested
on everything from Sony Playstations to AMD Opterons to IBM
mainframes. Heck, there are probably more than 70 machines running
CVS versions of PostgreSQL right this minute (Marc, any download
numbers to back this up?). More importantly, PostgreSQL gets tested
on a wide variety of real world tasks, and not some lab application or
some test scripts. Like I have mentioned several times before.
PostgreSQL gets tested by folks that put their actual data into the
beta versions and try it out. Even with this sort of testing,
however, bugs still make it into the release version. Even with a
large group of beta testers we simply can't test all of the possible
ways that the software might get used on every available platform.

If there is no procedure for PostgreSQL of this nature, then there
really needs to be. I am sure that MySQL must have something in place
like that. Their "Crash-Me" test suite shows (at least) that they have
put a large effort into testing.

Yow! Have you read the crash-me script. It's possible that they have
improved dramatically in the year or so since I last took a look at
them, but it used to be that MySQL's crash-me scripts were the worst
amalgamation of marketeering and poor relational theory ever conceived
by the human mind. Basically the crash-me scripts were nothing more
than an attempt to put MySQL's competition in the worst light
possible. Basically any time a competitor differed from MySQL an
error would be generated (despite the fact that it was very likely
that it was MySQL that was wrong).

MySQL even tried to pawn this single-process monstrosity off as a
"benchmark." What a laugh. It was a perfectly valid benchmark if
your database was designed to be used by one user at a time and one of
your biggest criteria was the time it took to create a valid
connection from a perl script.

PostgreSQL's regression tests (IMHO) are much better than MySQL's
crash-me scripts.

Jason

-----Original Message-----
From: Jason Earl [mailto:jason.earl@simplot.com]
Sent: Friday, June 20, 2003 4:43 PM
To: Dann Corbit
Cc: Jason Earl; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

"Dann Corbit" <DCorbit@connx.com> writes:

-----Original Message-----
From: Jason Earl [mailto:jason.earl@simplot.com]
Sent: Friday, June 20, 2003 3:32 PM
To: Dann Corbit
Cc: Jason Earl; The Hermit Hacker; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

"Dann Corbit" <DCorbit@connx.com> writes:

Why couldn't you just release the win32 version of 7.4

when it was

finished. If it takes an extra month then that just gives you
guys the chance to circulate *two* press releases.
The Native Win32 port is likely to make a big enough splash
all by itself.

A formal release needs a big testing effort. Two

separate releases

will double the work of validation.

There are several problems with that statement. The first is
that PostgreSQL's "testing effort" happens right here on this
mailing list.

That's not exactly reassuring. There is no regression test

that gets

formal acceptance?!

Yes, there are regression tests, and new tests get invented
all of the time whenever the real world finds new bugs.
Regression tests are excellent for making sure that you don't
make the same mistake twice, but they aren't a substitute for
handing the code over to actual end users.

After testing & QA, there is a beta period. You don't hand beta code
over to actual end users. In the corporate world it would be a clear
case of both negligence and incompetence.

The various PostgreSQL hackers code stuff up,
and we try and break it. There's very little /effort/
involved. People that want the new features go out on a limb
and start using them. If they don't work, then they bring it
up on the list. If they do work then very little gets said.

As it now stands Tom Lane is on the record as stating that
the new Win32 version isn't going to be ready for production
anyhow. If anything the Win32 version *should* get released
separately simply because we don't want people mistaking the
Win32 version as being up to the PostgreSQL teams high
standards. Those people that want the Win32 version to
become production ready are going to have to risk their
precious data. Otherwise, the Win32 version will likely
remain a second class citizen forever.

The fact of the matter is that the Win32 specific bits are
the parts that are likely to break in the new port. If
anything the Windows version will *benefit* from an earlier
*nix release because the *nix users will chase down the bugs
in the new PostgreSQL features. Once the *nix version is up
to 7.4.2 (or so) then a Windows release of 7.4.2 should allow
the PostgreSQL hackers to simply chase down Windows

specific problems.

Then using the same logic, the new Windows version should wait
indefinitely, since the *nix version will always be shaking

out bugs.

That's not true at all. Despite the excellent work by the
PostgreSQL team, and despite the beta testing that will be
done by volunteers, if history repeats itself, there will be
problems with version 7.4.0, even on platforms that have been
well supported by PostgreSQL forever. I am not saying that we
should hold off indefinitely on the Win32 port, I am simply
saying that it probably wouldn't hurt to shake out the normal
.0 release bugs before throwing the unique Win32 bugs into the mix.

My guess is that reported Win32 bugs are going blamed on the
Win32 specific bits at first no matter what happens. Unless
the bug can be demonstrated on a *nix version it will be
assumed that the problem is a shortcoming of the Win32
specific code. That's just common sense.

You are right that a new feature will add new bugs. I am saying that
the Win32 port is a new feature that will need a shakedown, but the
shakedown should occur in the testing and beta phase, like any other
feature.

Adding a new platform--especially a platform as diverse from
the rest of PostgreSQL's supported platforms as Windows--is
what adds the work. Testing the new platform is relatively
easy. All you need to do is to start using the Win32 version
with real live data.

That is not testing. Using the world as your beta team

seems to be a

business model used by a few software giants that is

largely frowned

upon. I would think that there is an opportunity to do things
differently. [Read 'properly'].

Hmm... I must have missed the huge corporation paying for in
house testing of PostgreSQL. In the Free Software world the
"beta team" is all of those people that need the new features
so badly that they are willing to risk their own data and
hardware testing it.

I don't see how this model can possibly succeed then. You can't just
hope that your end users will:
1. Exhaustively test
2. Accurately report the findings

You might not like the way that this
sounds, but in practice it works astoundingly well. Chances
are you can't name 25 pieces of commercial software that run
on the wide array of hardware platforms and OSes as
PostgreSQL, and PostgreSQL has a earned a well deserved
reputation for being a solid piece of software. Clearly the
PostgreSQL team is doing
*something* right.

I don't argue that PostgreSQL is a good piece of software. I happen to
like it very much and have been a staunch advocate for its use with our
commercial products as well as in house. What I am saying is that it
may be possible to improve the process.

If the corporate world knew that the only testing applied to PostgreSQL
was ad-hoc, I doubt that it would be accepted at all anywhere. The fact
that PostgreSQL does succeed shows that the installed base of users must
be highly intelligent and highly motivated.

We (at CONNX Solutions Inc.) have a formal release procedure that
includes many tens of thousands of automated tests using dozens of
different platforms. There are literally dozens of

machines (I would

guess 70 or so total) running around the clock for 7 days before we
even know if we have a release candidate. The QA team is distinct
from the development team, and if they say "FLOP!" the release goes
nowhere. No formal release until QA passes it.

And yet when you release the software your customers
invariably find bugs, don't they?

Our beta customers do help us to find bugs. Bugs reported by customers
for released products are extremely rare. The total issue count is 2495
in our bug tracking database (active since the late 1980's). There are
82 issues found by the customers in that list, and 7 with an issue ID
over 2000 (recent issues). Our code base is several hundred thousand
lines of code, and we have many thousands of customers world-wide. When
I first started here, testing was less rigorous, and largely done by the
programmers instead of separate testing teams. Since formal testing
procedures have been established, technical support incidents have gone
way down and quality has gone way up.

Don't get me wrong. I am all for testing, regression tests,
and such, but the fact of the matter is that there simply is
no way that a centralized authority could afford to really
test PostgreSQL on even a fraction of the supported platforms
and configurations. The way it stands now the PostgreSQL
teams gets the best testbed you could hope for (the world)
for the price of hosting a few web and FTP servers (thanks Marc).

PostgreSQL betas almost certainly gest tested on an order of
magnitude more systems than the 70 that you boast about.

Maybe it does. Maybe it doesn't. You have no way of knowing, since no
formal reporting procedure exists.

PostgreSQL gets tested on everything from Sony Playstations
to AMD Opterons to IBM mainframes. Heck, there are probably
more than 70 machines running CVS versions of PostgreSQL
right this minute (Marc, any download numbers to back this
up?).

If your count all the end-users workstations, our products have millions
of seats. We run on UNIX (Solaris, Linux, AIX, Tru64, HP/UX, etc.) and
OpenVMS and MVS and Win32 and OS/400 and others. As you can well
imagine, we *must* have an enormous testing effort.

More importantly, PostgreSQL gets tested on a wide
variety of real world tasks, and not some lab application or
some test scripts.

Spoken like a programmer. Yes, real world data *always* turns up things
that neither the testers nor the programmers imagined. But a huge and
comprehensive conformance testing effort will turn up 99% of the
problems.

Like I have mentioned several times
before. PostgreSQL gets tested by folks that put their actual
data into the beta versions and try it out. Even with this
sort of testing, however, bugs still make it into the release
version.

Bug cost as a function of discovery stage is exponential.
1. Discovered in design phase: nearly free to fix (designer sees bug,
designer fixes bug)
2. Discovered in unit test phase: very cheap to fix (programmer sees
bug, programmer fixes bug)
3. Discovered in integration test phase: inexpensive to fix (other
engineers become involved)
4. Discovered in beta test phase: expensive to fix (customers,
tech-support, sales, programmers, engineers involved)
5. Discovered in release: catastrophic cost to fix (as above, but now
every single customer must be upgraded, tens of thousands of dollars
lost, minimum -- possibly millions)

Even with a large group of beta testers we simply
can't test all of the possible ways that the software might
get used on every available platform.

100% code coverage is impossible.
Program proving is impossible.
0% defect code delivery is impossible.

But you should try to approach the ideal as closely as can be attained.

If there is no procedure for PostgreSQL of this nature, then there
really needs to be. I am sure that MySQL must have

something in place

like that. Their "Crash-Me" test suite shows (at least) that they
have put a large effort into testing.

Yow! Have you read the crash-me script. It's possible that
they have improved dramatically in the year or so since I
last took a look at them, but it used to be that MySQL's
crash-me scripts were the worst amalgamation of marketeering
and poor relational theory ever conceived by the human mind.

The tests are good tests. They cover a wide range of features and
functions and discover if you can cause permanent damage to a database
by simply performing end-user queries. The scripts are a bit hokey, but
it isn't all that difficult to get them to run.

Basically the crash-me scripts were nothing more than an
attempt to put MySQL's competition in the worst light
possible.

I disagree. In fact, in their matrix, PostgreSQL looks remarkably good.
In fact, I would choose it over MySQL, if the only examination made was
of the information contained in the matrix (but nobody would really
drive a decision based on that data alone).

Basically any time a competitor differed from
MySQL an error would be generated (despite the fact that it
was very likely that it was MySQL that was wrong).

This is unfair and untrue. (I have no connection whatsoever with the
MySQL group, BTW).

MySQL even tried to pawn this single-process monstrosity off
as a "benchmark." What a laugh. It was a perfectly valid
benchmark if your database was designed to be used by one
user at a time and one of your biggest criteria was the time
it took to create a valid connection from a perl script.

You can call it a conformance benchmark. It is not touted as a
performance benchmark. And nobody would fall for it if it were, since
it does not contain time information.

PostgreSQL's regression tests (IMHO) are much better than
MySQL's crash-me scripts.

They are less thorough in coverage, but not too bad.

Here is what I suggest:

PostgreSQL has an excellent programming team. Why not try to recruit a
similar testing team? I think it would strongly differentiate the tool
set from similar free stuff.

Perhaps all that is needed is some sort of automated, formal reporting
procedure. For example, a large test set might be created that runs a
thorough regression feature list. When the test completes, a data file
is emailed to some central repository, parsed, and stored in a database.

If the end-users can simply start some shell script and take off for the
weekend, then it would be possible to collect a large volume of data.

Jason Earl <jason.earl@simplot.com> writes:

The fact of the matter is that the Win32 specific bits are the parts
that are likely to break in the new port.

Actually, what scares me about this is the probability that the Win32
port will break other platforms. The changes look to be invasive enough
to create a nontrivial risk of that.

For comparison, look at the CVS history of the recent efforts to support
IPv6 connections. Those patches have broken both IPv4 and Unix-socket
connections at different times, and are still a source of ongoing build
problems on some platforms, plus who-knows-what problems yet to be found
on platforms that aren't used by the bleeding-edge-CVS folk. I predict
that the tweaks needed to support Windows' lack of a fork() primitive
will be far worse.

BTW, I would not approve of a response along the lines of "can't you
#ifdef to the point that there are no code changes in the Unix builds?"
No you can't, unless you want to end up with an unmaintainable mess
of #ifdef spaghetti. The thing that makes this hard is the tradeoff
between making the code readable and maintainable (which requires
sharing as much code as possible across platforms) vs isolating
platform-specific considerations. Programming at this level is not
a science but an art form, and it's very hard to get it right the first
time --- especially when none of us have access to all the platforms
that the code must ultimately work on.

regards, tom lane

"Dann Corbit" <DCorbit@connx.com> writes:

If there is no procedure for PostgreSQL of this nature, then there
really needs to be.

Are you volunteering to create it? Step right up.

I am sure that MySQL must have something in place
like that. Their "Crash-Me" test suite shows (at least) that they have
put a large effort into testing.

...ROTFL... Crash-Me is not a regression test. It is a marketing
effort.

regards, tom lane

Alvaro Herrera <alvherre@dcc.uchile.cl> writes:

It would be nice to have a system which could receive a patch and
compile and verify that it passes the tests before it goes to Bruce's
queue; or compile on multiple platforms to check for portability
problems, for example.

It happens not infrequently that Bruce commits some patch that fails the
regression tests. Whereupon he gets chewed out by whoever notices it
first :-). I've been guilty of the same on occasion, even though I try
hard to avoid it. If the regression tests took two seconds to run I'm
sure we'd both set up scripts to ensure we *never* commit without
regression testing first. On the other hand, if they took a week to run
you can be damn sure that most commits would go in without any
regression testing. I think that our existing tests are a fairly happy
medium --- they catch most stuff, and the stuff they don't catch is in
my opinion stuff that an automated test is unlikely to catch. (I do add
things to the regression tests whenever something shows up that looks
like it should have been caught.)

Another point is that passing on one platform doesn't ensure passing on
another. Here we really rely on the willingness of the pghackers
community to update to CVS tip regularly and run the regression tests
when they do. Again, tests that take a couple minutes to run are ideal;
if they took a week then the uptake would drop to zero, and we'd not be
ahead.

regards, tom lane

Jason Earl <jason.earl@simplot.com> writes:

Hmm... I must have missed the huge corporation paying for in house
testing of PostgreSQL. In the Free Software world the "beta team" is
all of those people that need the new features so badly that they are
willing to risk their own data and hardware testing it.

I don't have a lot of faith in huge automated test efforts. They're
great at ensuring you don't make the same mistakes you made once before,
but in my experience the nastiest bugs are the ones you haven't seen
before and would never in a million years have dreamed to test for.
Thus, the best test team is a bunch of people doing unplanned things
with the software, on a wide variety of platforms...

regards, tom lane

Tom Lane wrote:

Alvaro Herrera <alvherre@dcc.uchile.cl> writes:

It would be nice to have a system which could receive a patch and
compile and verify that it passes the tests before it goes to Bruce's
queue; or compile on multiple platforms to check for portability
problems, for example.

*snip*

Another point is that passing on one platform doesn't ensure passing on
another. Here we really rely on the willingness of the pghackers
community to update to CVS tip regularly and run the regression tests
when they do. Again, tests that take a couple minutes to run are ideal;
if they took a week then the uptake would drop to zero, and we'd not be
ahead.

Have you considered something similar to the Mozilla tinderbox approach
where you have a daemon checkout the cvs, compile, run regression tests,
and report a status or be able to report a status?

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Friday, June 20, 2003 8:36 PM
To: Dann Corbit
Cc: Jason Earl; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

"Dann Corbit" <DCorbit@connx.com> writes:

If there is no procedure for PostgreSQL of this nature, then there
really needs to be.

Are you volunteering to create it? Step right up.

No. And as an outsider, I rather doubt if any procedures I developed
would be taken very seriously. If such procedures are to be developed,
I suspect that they will have to be developed from within if they are to
be successful.

This would be a good start:

A. Combine:
1. Your regression test
2. Crashme (or some rough equivalent if you don't like it)
3. The NIST validation test suite
B. Automate:
1. Installation of the tests
2. Execution of the tests
3. Transfer of the test results to a repository
4. Analysis of the test results
C. Assign:
1. Criteria for acceptance of a build for release
2. Authority for acceptance of a build for release
3. Delegation rules for issue resolution
4. Procedures for issue resolution

I am sure that MySQL must have something in place
like that. Their "Crash-Me" test suite shows (at least) that they
have put a large effort into testing.

...ROTFL... Crash-Me is not a regression test. It is a
marketing effort.

Let's see...
Their marketing effort checks for STANDARDS conformance against over
several hundred distinct, important properties.
Their marketing effort checks for a number of interesting and valuable
extensions.
Their marketing effort checks for system safety in a manner that is
better than anything I have ever seen from a commercial vendor.

And the PostgreSQL regression test is superior in what ways?

Look at this:
http://www.mysql.com/information/crash-me.php?mysql_4_1=on&amp;postgres=on

Their marketing effort makes PostgreSQL look superior to MySQL in most
areas. If it is a marketing effort, then we must applaud them for their
honesty.

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Friday, June 20, 2003 8:58 PM
To: Jason Earl
Cc: Dann Corbit; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

Jason Earl <jason.earl@simplot.com> writes:

Hmm... I must have missed the huge corporation paying for in house
testing of PostgreSQL. In the Free Software world the

"beta team" is

all of those people that need the new features so badly

that they are

willing to risk their own data and hardware testing it.

I don't have a lot of faith in huge automated test efforts.
They're great at ensuring you don't make the same mistakes
you made once before, but in my experience the nastiest bugs
are the ones you haven't seen before and would never in a
million years have dreamed to test for.

This is true if and only if the test design is poor.

Thus, the best test
team is a bunch of people doing unplanned things with the
software, on a wide variety of platforms...

That is the worst possible test plan. It totally lacks organization and
there is no hint to define when the feature set has been covered. Ad
hoc testing is a useful addition, but it cannot replace all the standard
tests that have been used by the industry for decades.

If you run literally hundreds of tests designed to ensure that your
product conforms to ANSI/ISO standards then the bugs that are missed
will be few and far between. Unless you are bad at designing tests.

Designing tests is busywork. Desiging tests is boring. Nobody wants to
design tests, let alone interpret the results and define correct
baselines. But testing is very, very important.

--- Dann Corbit <DCorbit@connx.com> wrote:

Why couldn't you just release the win32 version of 7.4
when it was finished.

I agree. Don't delay *nix release because of win32 port is not ready. To many
users win32 port is of marginal importance anyway.

__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com

Thomas Swan <tswan@idigx.com> writes:

Have you considered something similar to the Mozilla tinderbox approach
where you have a daemon checkout the cvs, compile, run regression tests,
and report a status or be able to report a status?

Tinderbox is pretty cool. Who wants to set it up?

regards, tom lane

"Dann Corbit" <DCorbit@connx.com> writes:

...ROTFL... Crash-Me is not a regression test. It is a
marketing effort.

Their marketing effort checks for STANDARDS conformance against over
several hundred distinct, important properties.

If you'd not spelled STANDARDS in caps I'd not have taken the trouble to
respond ... but I suggest you stop to count exactly how many of their
bullet points are actually grounded in the SQL standard, how many are
not, and how many are in fact counter to spec but agree with MySQL's
deviations from spec (of course those show as green for MySQL's version
of reality, and as "failures" for spec-compliant databases).

I have been through crash-me in some detail, and it left a very bad
taste in my mouth. Don't bother holding it up as an example of good
practice.

regards, tom lane

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Friday, June 20, 2003 9:19 PM
To: Dann Corbit
Cc: Jason Earl; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

"Dann Corbit" <DCorbit@connx.com> writes:

...ROTFL... Crash-Me is not a regression test. It is a
marketing effort.

Their marketing effort checks for STANDARDS conformance

against over

several hundred distinct, important properties.

If you'd not spelled STANDARDS in caps I'd not have taken the
trouble to respond

Sorry for shouting.

... but I suggest you stop to count
exactly how many of their bullet points are actually grounded
in the SQL standard, how many are not, and how many are in
fact counter to spec but agree with MySQL's deviations from
spec (of course those show as green for MySQL's version of
reality, and as "failures" for spec-compliant databases).

I have been through crash-me in some detail, and it left a
very bad taste in my mouth. Don't bother holding it up as an
example of good practice.

Every single test in their list is interesting and useful.

I see several hundred things which I recognize as part of the ANSI/ISO
SQL Standard.

I have set up and run the tests. I did not go into great detail (as you
have done) to ensure that they were really testing what they claimed to
test and that correct interpretation of the results was made in each
case.

If they have done something underhanded, then they should be called out
onto the carpet for it. In any case, the general outline of what they
are doing is a very good idea. If it can be improved upon, then that
would be an excellent idea.

On Fri, Jun 20, 2003 at 09:25:08PM -0700, Dann Corbit wrote:

Citing Tom Lane:

I have been through crash-me in some detail, and it left a
very bad taste in my mouth. Don't bother holding it up as an
example of good practice.

Every single test in their list is interesting and useful.

At least on the version I just saw there are several results with
Postgres that are weird (table names > 500 chars?). Other things tested
are clearly wrong (things that are = NULL, dates like '00-00-0000');
results for Postgres that are wrong probably because they are trying a
weird syntax. Etc.

Things like that drive the credibility of the whole thing to the floor.
Maybe something like this should exist for Postgres, but it's not
crash-me. Maybe the NIST compliance test is adequate.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"La conclusion que podemos sacar de esos estudios es que
no podemos sacar ninguna conclusion de ellos" (Tanenbaum)

-----Original Message-----
From: Alvaro Herrera [mailto:alvherre@dcc.uchile.cl]
Sent: Friday, June 20, 2003 10:00 PM
To: Dann Corbit
Cc: Tom Lane; Jason Earl; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

On Fri, Jun 20, 2003 at 09:25:08PM -0700, Dann Corbit wrote:

Citing Tom Lane:

I have been through crash-me in some detail, and it left a
very bad taste in my mouth. Don't bother holding it up as an
example of good practice.

Every single test in their list is interesting and useful.

At least on the version I just saw there are several results
with Postgres that are weird (table names > 500 chars?).

It does get silly at a point, but I have seen systems with 128
characters for table names, column names, etc. Some people seem to like
it. Not me. Too much typing.

Other things tested are clearly wrong (things that are =
NULL,

Sounds like testing for the existence of a bug.
X = NULL
X <= NULL
X >= NULL
Etc. must always test false, regardless of the contents of X. Test for
equality with NULL is a conformance error if NULL == NULL returns true.

dates like '00-00-0000');

Not sure what that might even mean.

results for Postgres that are
wrong probably because they are trying a weird syntax. Etc.

Things like that drive the credibility of the whole thing to
the floor. Maybe something like this should exist for
Postgres, but it's not crash-me. Maybe the NIST compliance
test is adequate.

So far, I have seen three problems pointed out (out of 600+ tests).
That's 0.5% defects. Why not just drop the stupid tests, or bend them
to test for what they ought to be testing.

Besides those three, what other tests are bogus and why?

We (at CONNX Solutions Inc.) have a formal release procedure that
includes many tens of thousands of automated tests using dozens of
different platforms. There are literally dozens of machines (I would
guess 70 or so total) running around the clock for 7 days before we even
know if we have a release candidate. The QA team is distinct from the
development team, and if they say "FLOP!" the release goes nowhere. No
formal release until QA passes it.

PostgreSQL has a comprehensive regression suite that is run by the
developers all the time...

If there is no procedure for PostgreSQL of this nature, then there
really needs to be. I am sure that MySQL must have something in place
like that. Their "Crash-Me" test suite shows (at least) that they have
put a large effort into testing.

No, it means they've put a crap effort into trying to make other databases
look bad...

Chris

-----Original Message-----
From: Christopher Kings-Lynne [mailto:chriskl@familyhealth.com.au]
Sent: Friday, June 20, 2003 10:14 PM
To: Dann Corbit
Cc: Jason Earl; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

We (at CONNX Solutions Inc.) have a formal release procedure that
includes many tens of thousands of automated tests using dozens of
different platforms. There are literally dozens of

machines (I would

guess 70 or so total) running around the clock for 7 days before we
even know if we have a release candidate. The QA team is distinct
from the development team, and if they say "FLOP!" the release goes
nowhere. No formal release until QA passes it.

PostgreSQL has a comprehensive regression suite that is run
by the developers all the time...

If you mean the one that comes with PostgreSQL, then I think the MySQL
test is better. The PostgreSQL test seems to focus more on extensions
than anything else.

If there is no procedure for PostgreSQL of this nature, then there
really needs to be. I am sure that MySQL must have

something in place

like that. Their "Crash-Me" test suite shows (at least) that they
have put a large effort into testing.

No, it means they've put a crap effort into trying to make
other databases look bad...

It does not achieve that goal.

Most of the criticism leveled at their efforts sound like fearful hand
waving to me. True, I have not studied the test as carefully as others
have. But the PostgreSQL test is not superior to the MySQL test. I
have put considerable effort into the PostgreSQL regression test. We
achieved 100% success on the Win32 platform, including dynamic loading
of functions.

I don't have a lot of faith in huge automated test efforts. They're
great at ensuring you don't make the same mistakes you made once before,
but in my experience the nastiest bugs are the ones you haven't seen
before and would never in a million years have dreamed to test for.
Thus, the best test team is a bunch of people doing unplanned things
with the software, on a wide variety of platforms...

Which is why I never use a .0 release of PostgreSQL :)

Chris

Things like that drive the credibility of the whole thing to the floor.
Maybe something like this should exist for Postgres, but it's not
crash-me. Maybe the NIST compliance test is adequate.

Plus I belive the RedHat people are getting PostgreSQL through the NIST
compliance tests at the moment...I'd love to see MySQL pass them...

Chris

Sounds like testing for the existence of a bug.
X = NULL
X <= NULL
X >= NULL
Etc. must always test false, regardless of the contents of X. Test for
equality with NULL is a conformance error if NULL == NULL returns true.

They should all return NULL, not false...

dates like '00-00-0000');

Yes, that's MySQL's idea of a NULL date. In fact, it's exactly what you
get when you insert a NULL date into a NOT NULL column - hooray the test
proves that MySQL accepts NULL values in NOT NULL columns...

Chris

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

Maybe the NIST compliance test is adequate.

Plus I belive the RedHat people are getting PostgreSQL through the NIST
compliance tests at the moment...I'd love to see MySQL pass them...

FWIW, the first pass of those tests is complete, and it turned up
exactly one bug that we didn't already know of (the
outer-level-aggregate bizarrity that I fixed last week ... which MySQL
wouldn't be subject to since they haven't got subselects ...)

The work is not done, because there are some tests that couldn't be run
because they were blocked by known noncompliances (such as lack of
updatable views). But I'm not getting a sense that we will learn a
whole lot from the NIST tests.

Further details will be published soon...

regards, tom lane

If you mean the one that comes with PostgreSQL, then I think the MySQL
test is better. The PostgreSQL test seems to focus more on extensions
than anything else.

What the? It tests no extensions. The extensions have their own
regression tests.

Most of the criticism leveled at their efforts sound like fearful hand
waving to me. True, I have not studied the test as carefully as others
have. But the PostgreSQL test is not superior to the MySQL test. I
have put considerable effort into the PostgreSQL regression test. We
achieved 100% success on the Win32 platform, including dynamic loading
of functions.

Notice that it tests absulutely no parallel functionality, whereas
PostgreSQL tests things in parallel to check for concurrency problems:

"Note that this benchmark is single threaded, so it measures the minimum
time for the operations. We plan to in the future add a lot of
multi-threaded tests to the benchmark suite. "

It's said that for at least 4 years now.

Crash-me has nothing to do with testing, it jsut checks to see what
features a db supports:

"crash-me tries to determine what features a database supports and what
its capabilities and limitations are by actually running queries. For
example, it determines:

What column types are supported
How many indexes are supported
What functions are supported
How big a query can be
How big a VARCHAR column can be"

Obviously it has nothing to do with can I index every type in the system,
can I use the index to look up a set of test values, etc., etc.

Chris

"Dann Corbit" <DCorbit@connx.com> writes:

Look at this:
http://www.mysql.com/information/crash-me.php?mysql_4_1=on&amp;postgres=on

This looks a little cleaner than the last time I looked at it (more than
three years ago), but it's still fundamentally a marketing effort. It
is not an exercise in spec compliance measurement, because there are
hundreds of bullet points that all look exactly alike, whether they are
measuring spec-required elements, random vendor extensions to the spec,
or spec violations. To take just one example of the latter,
"Calculate 1--1" is still shown with a green star for MySQL and a
failure for Postgres, when a more correct reading would be "Fails to
recognize SQL-standard -- comment syntax" for MySQL. And yes, they were
called out on this three years ago, and no they haven't fixed the entry
since then. I should believe that there is any good faith on their
part?

For another example, take a close look at the "Quoting" section, which
purports to measure compliance to the spec's ideas about how to quote
an identifier. Postgres accepts double-quoted identifiers per spec,
including doubled double quotes per spec, and rejects bracketed or
backquoted identifiers per spec. MySQL is apparently spec compliant on
just one of those four points. Curious that they manage to end up with
a better looking display than us in this section; in particular note
that Postgres is specifically claimed *not* to handle double-quoted
identifiers. (Memory is fuzzy after three years, but IIRC when you look
at the actual test code being used, it tests more than whether double
quoted identifiers are allowed, and really is failing us on some quite
unrelated detail.)

Another point worth mentioning is that most of the numerical limits
shown in the table have nothing to do with actual server limits, but
with random limitations of their test process. For instance, I'm not
sure what "max index part length 235328" really means, but I am pretty
sure it's got nothing to do with the Postgres server. Or look at
"constant string size in SELECT 16777207" ... nope, there's no such
limit. (If they'd put a "+" in there then it'd be okay, but no.)
I still remember watching crash-me trying to measure the max query
length of Postgres 7.0: the crashme client process dumped core before
Postgres did, after which the controlling script announced that we
weren't crash-safe.

So far, I have seen three problems pointed out (out of 600+ tests).

These are the high spots from three-year-old memories. Do you really
want a detailed analysis? A quick look at their table recalls plenty
of bogosity to my mind.

A last point is that this table is comparing MySQL 4.1 (bleeding edge
alpha release) against PG 7.2 (one full major release behind the times).
While I cannot really blame the MySQL guys for not being up-to-the-
minute on everyone else's releases, this does emphasize the key point,
namely that this isn't a fair comparison run by disinterested parties
but a marketing effort of, by, and for MySQL.

regards, tom lane

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Friday, June 20, 2003 11:47 PM
To: Dann Corbit
Cc: Jason Earl; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

"Dann Corbit" <DCorbit@connx.com> writes:

Look at this:

http://www.mysql.com/information/crash-me.php?mysql_4_1=on&amp;postgres=on

This looks a little cleaner than the last time I looked at it
(more than three years ago), but it's still fundamentally a
marketing effort. It is not an exercise in spec compliance
measurement, because there are hundreds of bullet points that
all look exactly alike, whether they are measuring
spec-required elements, random vendor extensions to the spec,
or spec violations. To take just one example of the latter,
"Calculate 1--1" is still shown with a green star for MySQL
and a failure for Postgres, when a more correct reading would
be "Fails to recognize SQL-standard -- comment syntax" for
MySQL. And yes, they were called out on this three years
ago, and no they haven't fixed the entry since then. I
should believe that there is any good faith on their part?

For another example, take a close look at the "Quoting"
section, which purports to measure compliance to the spec's
ideas about how to quote an identifier. Postgres accepts
double-quoted identifiers per spec, including doubled double
quotes per spec, and rejects bracketed or backquoted
identifiers per spec. MySQL is apparently spec compliant on
just one of those four points. Curious that they manage to
end up with a better looking display than us in this section;
in particular note that Postgres is specifically claimed
*not* to handle double-quoted identifiers. (Memory is fuzzy
after three years, but IIRC when you look at the actual test
code being used, it tests more than whether double quoted
identifiers are allowed, and really is failing us on some
quite unrelated detail.)

Another point worth mentioning is that most of the numerical
limits shown in the table have nothing to do with actual
server limits, but with random limitations of their test
process. For instance, I'm not sure what "max index part
length 235328" really means, but I am pretty sure it's got
nothing to do with the Postgres server. Or look at "constant
string size in SELECT 16777207" ... nope, there's no such
limit. (If they'd put a "+" in there then it'd be okay, but
no.) I still remember watching crash-me trying to measure the
max query length of Postgres 7.0: the crashme client process
dumped core before Postgres did, after which the controlling
script announced that we weren't crash-safe.

So far, I have seen three problems pointed out (out of 600+ tests).

These are the high spots from three-year-old memories. Do
you really want a detailed analysis? A quick look at their
table recalls plenty of bogosity to my mind.

A last point is that this table is comparing MySQL 4.1
(bleeding edge alpha release) against PG 7.2 (one full major
release behind the times). While I cannot really blame the
MySQL guys for not being up-to-the- minute on everyone else's
releases, this does emphasize the key point, namely that this
isn't a fair comparison run by disinterested parties but a
marketing effort of, by, and for MySQL.

It seems pretty clear that there are warts on the Crashme test.
Perhaps 70% or so is truly useful. Maybe the useful subset could be
approximated or modified to be useful as a general tool set.

Not too surprising that a commercial enterprise tries to bend the facts
in their favor a bit.

Some other stuff worth note:
http://osdb.sourceforge.net/
http://sourceforge.net/projects/osdldbt (looks like someone has put a
bunch of PostgreSQL effort into it.
http://sourceforge.net/projects/ltp/ (DOTS)
http://www.mysql.com/portal/software/item-222.html (I won't mention
where it's from)
ftp://ftp.cs.wisc.edu/OO7/

Win32 specific, but has source code:
http://www.mipt.sw.ru/en/install/ots/ (ODBC testing)
http://www.mipt.sw.ru/en/install/ats/ (ADO testing)
Some other interesting stuff is found there too...

Test tools links:
http://www.softwareqatest.com/qattls1.html
http://www.aptest.com/resources.html

----- Original Message -----
From: "Tom Lane" <tgl@sss.pgh.pa.us>
To: "Christopher Kings-Lynne" <chriskl@familyhealth.com.au>
Cc: "Alvaro Herrera" <alvherre@dcc.uchile.cl>; "Dann Corbit"
<DCorbit@connx.com>; "Jason Earl" <jason.earl@simplot.com>;
"PostgreSQL-development" <pgsql-hackers@postgresql.org>
Sent: Saturday, June 21, 2003 8:33 AM
Subject: Re: [HACKERS] Two weeks to feature freeze

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

Maybe the NIST compliance test is adequate.

Plus I belive the RedHat people are getting PostgreSQL through the NIST
compliance tests at the moment...I'd love to see MySQL pass them...

FWIW, the first pass of those tests is complete, and it turned up
exactly one bug that we didn't already know of (the
outer-level-aggregate bizarrity that I fixed last week ... which MySQL
wouldn't be subject to since they haven't got subselects ...)

The work is not done, because there are some tests that couldn't be run
because they were blocked by known noncompliances (such as lack of
updatable views). But I'm not getting a sense that we will learn a
whole lot from the NIST tests.

As far as I can see NIST (http://www.itl.nist.gov/div897/ctg/sql_form.htm)
tests are used *only* for testing SQL92 conformance.
Latest available test suite version 6, dated 12/1996.
Are you using about 6-7 years old test suite?
Perhaps I am wrong, I would be happy If someone could point me up-to-date
info about NIST conformance testing.

regards,
Alvis Tunkelis

Dann Corbit writes:

So far, I have seen three problems pointed out (out of 600+ tests).
That's 0.5% defects. Why not just drop the stupid tests, or bend them
to test for what they ought to be testing.

The problem with crashme is that it tells you nothing of practical value.
It doesn't tell you whether PostgreSQL works right. It doesn't tell you
whether PostgreSQL works well. It doesn't tell you whether PostgreSQL
conforms to some standard. It doesn't even tell whether PostgreSQL is
compatible to some other product. The only thing that you could possibly
get out of fixing crashme is to look better in crashme. And for the above
reasons, there is little interest in working on that.

--
Peter Eisentraut peter_e@gmx.net

PostgreSQL has a comprehensive regression suite that is run
by the developers all the time...

If you mean the one that comes with PostgreSQL, then I think the MySQL
test is better. The PostgreSQL test seems to focus more on extensions
than anything else.

I would be happy to make additions to the regression tests if you can
give me a list of items that are missing. I've looked and didn't see
anything obvious aside from inter-connection testing. Yes, tests run in
parallel on multiple connections, but there is no interaction between
since there is not a method of controlling the timing at the moment.

--
Rod Taylor <rbt@rbt.ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

Thomas Swan writes:

Have you considered something similar to the Mozilla tinderbox approach
where you have a daemon checkout the cvs, compile, run regression tests,
and report a status or be able to report a status?

Even if you could achieve near complete coverage of the platforms,
platform versions, and auxilliary software versions and combinations that
PostgreSQL runs with, in most cases, something breaks on a new
version or combination of these things.

--
Peter Eisentraut peter_e@gmx.net

On Fri, 20 Jun 2003, Josh Berkus wrote:

Ultimately, this is one of those "technical" vs. "marketing" questions
... whether to release now with a bunch of back-end features that the
current users want, or to release later and include the features that we
said were going to be in 7.4. And PostgreSQL is a technical project,
not a marketing one.

Technical or Marketing, I think ppl are putting too much emphasis on
'visible features' and not enough on the 'not so visible' ones ...
improvements to both performance and footprint are massive changes, but
they are more difficult to 'market', then, say, adding schemas was ...

On Fri, 20 Jun 2003, Tom Lane wrote:

Time was that we had a major release every 3 or 4 months. As the
project matures I think it's appropriate for the cycle to get slower: a
lot of low-hanging fruit is gone, so we have larger jobs to tackle, plus
users are using PG for larger databases and don't want to face
major-version changes too often. But I don't want it to get to be a
year on average between releases, at least not yet. 8 or 9 months seems
reasonable, and by that standard we're overdue.

Note that with how we've been releasing 'minors' on v7.3.x semi-regularly,
slippage isn't *as* big an issue as it could have been ...

On Fri, 20 Jun 2003, Jason Earl wrote:

Heck, there are probably more than 70 machines running
CVS versions of PostgreSQL right this minute (Marc, any download
numbers to back this up?).

Unfortunately, most ppl testing would be using CVS or CVSup, which don't
(or, at least, I haven't been able to find?) log such ..

download wise, through the FTP site, maybe 20 downloads since the 4th of
June of the snapshot ...

On Fri, 20 Jun 2003, Dann Corbit wrote:

Hmm... I must have missed the huge corporation paying for in
house testing of PostgreSQL. In the Free Software world the
"beta team" is all of those people that need the new features
so badly that they are willing to risk their own data and
hardware testing it.

I don't see how this model can possibly succeed then. You can't just
hope that your end users will:
1. Exhaustively test
2. Accurately report the findings

But it does, and has for 10 years now ...

Our beta customers do help us to find bugs. Bugs reported by customers
for released products are extremely rare.

Check the past archives for the mailing lists ... our "bugs reported by
end users for released products" is extremely rare also, and *generally*
is a result of them doing something that nobody had thought to test for
before ...

Spoken like a programmer. Yes, real world data *always* turns up things
that neither the testers nor the programmers imagined. But a huge and
comprehensive conformance testing effort will turn up 99% of the
problems.

And ours do ... I don't believe I can recall us having a release where
we've had a stream of problem reports come flying in afterwards ... we
might get one or two from ppl that have hit a 'never before seen' bug,
that generally gets fixed very quickly ...

100% code coverage is impossible.
Program proving is impossible.
0% defect code delivery is impossible.

But you should try to approach the ideal as closely as can be attained.

And we do ...

The tests are good tests. They cover a wide range of features and
functions and discover if you can cause permanent damage to a database
by simply performing end-user queries. The scripts are a bit hokey, but
it isn't all that difficult to get them to run.

Well, if you would like to volunteer to run them against PostgreSQL, and
let us know what fails, we can let you know why said test is wrong in the
first place ... we've been through crash-me several times before, and
'fixing crash-me' was more work then it was worth ...

Basically any time a competitor differed from
MySQL an error would be generated (despite the fact that it
was very likely that it was MySQL that was wrong).

This is unfair and untrue. (I have no connection whatsoever with the
MySQL group, BTW).

Been there, done that ... even tried to get changes made to make the tests
more accurate ... it was like trying to move a mountain ...

PostgreSQL has an excellent programming team. Why not try to recruit a
similar testing team? I think it would strongly differentiate the tool
set from similar free stuff.

Are you volunteering? We already have a testing team we're happy with,
but if you would like to extend it with your resources, please feel free
to join in ...

Peter Eisentraut <peter_e@gmx.net> writes:

Thomas Swan writes:

Have you considered something similar to the Mozilla tinderbox approach
where you have a daemon checkout the cvs, compile, run regression tests,
and report a status or be able to report a status?

Even if you could achieve near complete coverage of the platforms,
platform versions, and auxilliary software versions and combinations that
PostgreSQL runs with, in most cases, something breaks on a new
version or combination of these things.

Still, whenever we're doing something that interacts at all with the OS,
it seems we get breakages that don't show in the original author's
testing, but only pop up days to months later when some beta tester
tries the code on platform P or using option Q. The current
difficulties with the IPv6 patches are a fine case in point.
If we could get feedback more easily about whether a proposed patch
compiles and passes regression on a variety of platforms, we could
reduce the pain involved by a great deal, simply because the problems
could be fixed while the code is still fresh in mind.

I don't think there is any company involved with Postgres that is
willing to commit the resources to run a Mozilla-style tinderbox setup
singlehanded. But I wonder whether we couldn't set up something that is
community-based: get a few dozen people with different platforms to
volunteer to check the code regularly on their own machines. I'm
imagining a cron job that fires daily in the wee hours, pulls the latest
CVS tip, does "make distclean; configure; make; make check", and mails
the results to someplace that puts 'em up on our website.

It's possible that we could adapt the tinderbox software to work this
way, but even if we had to write our own, it seems like a fairly simple
task. And it'd give *much* better feedback on porting problems than we
have now. Sure, there will always be corner cases you don't catch,
but the first rule of testing is the sooner you find a bug the cheaper
it is to fix.

regards, tom lane

--On Saturday, June 21, 2003 11:43:17 -0400 Tom Lane <tgl@sss.pgh.pa.us>
wrote:

Peter Eisentraut <peter_e@gmx.net> writes:

Thomas Swan writes:

Have you considered something similar to the Mozilla tinderbox approach
where you have a daemon checkout the cvs, compile, run regression tests,
and report a status or be able to report a status?

Even if you could achieve near complete coverage of the platforms,
platform versions, and auxilliary software versions and combinations that
PostgreSQL runs with, in most cases, something breaks on a new
version or combination of these things.

Still, whenever we're doing something that interacts at all with the OS,
it seems we get breakages that don't show in the original author's
testing, but only pop up days to months later when some beta tester
tries the code on platform P or using option Q. The current
difficulties with the IPv6 patches are a fine case in point.
If we could get feedback more easily about whether a proposed patch
compiles and passes regression on a variety of platforms, we could
reduce the pain involved by a great deal, simply because the problems
could be fixed while the code is still fresh in mind.

I don't think there is any company involved with Postgres that is
willing to commit the resources to run a Mozilla-style tinderbox setup
singlehanded. But I wonder whether we couldn't set up something that is
community-based: get a few dozen people with different platforms to
volunteer to check the code regularly on their own machines. I'm
imagining a cron job that fires daily in the wee hours, pulls the latest
CVS tip, does "make distclean; configure; make; make check", and mails
the results to someplace that puts 'em up on our website.

It's possible that we could adapt the tinderbox software to work this
way, but even if we had to write our own, it seems like a fairly simple
task. And it'd give *much* better feedback on porting problems than we
have now. Sure, there will always be corner cases you don't catch,
but the first rule of testing is the sooner you find a bug the cheaper
it is to fix.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

I'm willing to run such a job on UnixWare 7.1.3 and OpenUnix 8, as well
as FreeBSD 4.8

--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749

<alvis@piladzi-2.biz> writes:

As far as I can see NIST (http://www.itl.nist.gov/div897/ctg/sql_form.htm)
tests are used *only* for testing SQL92 conformance.
Latest available test suite version 6, dated 12/1996.
Are you using about 6-7 years old test suite?
Perhaps I am wrong, I would be happy If someone could point me up-to-date
info about NIST conformance testing.

AFAIK, the NIST abandoned that project years ago, so there isn't any
more-up-to-date test suite available. Not sure this is a big problem,
since SQL92 is not a moving target. It'd be nice if they had kept
working and developed tests for the non-entry-level features, but
they didn't.

regards, tom lane

On Fri, Jun 20, 2003 at 10:04:09PM -0700, Dann Corbit wrote:

On Fri, Jun 20, 2003 at 09:25:08PM -0700, Dann Corbit wrote:

Citing Tom Lane:

I have been through crash-me in some detail, and it left a
very bad taste in my mouth. Don't bother holding it up as an
example of good practice.

Every single test in their list is interesting and useful.

At least on the version I just saw there are several results
with Postgres that are weird (table names > 500 chars?).

It does get silly at a point, but I have seen systems with 128
characters for table names, column names, etc. Some people seem to like
it. Not me. Too much typing.

I meant that the real limit on 7.2 was much lower than that unless they
twiddled with sources at compile time (there's no configure switch for
that AFAIR). Maybe 31 or 63 chars, I don't remember. Do you really
trust the rest of the test seeing that they came up with a clearly wrong
answer in such a simple test?

They can't even "make vacuum run reliably" on 7.1. See the performance
test. Maybe they want to test 7.3 with lazy vacuum in place. Why don't
they do that? 7.1 is already 2 years old.

Other things tested are clearly wrong (things that are =
NULL,

Sounds like testing for the existence of a bug.
X = NULL
X <= NULL
X >= NULL
Etc. must always test false, regardless of the contents of X. Test for
equality with NULL is a conformance error if NULL == NULL returns true.

You see, you are saying "sounds like they are testing". What does the
code actually test? Which is the right behaviour? Which behaviour
gets the green point, MySQL's or the right one? There are lots of
things like this; I don't want to waste my time actually reading the
code to see what the correct answer for each test is.

About the 1--1 thing Tom mentioned: be aware that Postgres happily
accepts the correct 1 - -1 expression, but also correctly fails to
"calculate" 1--1. Which one gets the green point? Of course it's the
non-compliant one.

Also they don't test things they don't support. Is there a test for
subselects? What about concurrency? Transactional issues? What about
performance when they have their "transaction support" enabled?

So far, I have seen three problems pointed out (out of 600+ tests).
That's 0.5% defects. Why not just drop the stupid tests, or bend them
to test for what they ought to be testing.

There's already a mechanism for testing inside Postgres. Maybe more
tests are needed, but crash-me offers no real value.

I just became aware that the NIST test suite is quite old. Maybe what's
needed is to expand it to SQL3 to develop a way of measuring the
compliance level. But the cost of doing that is probably prohibitive.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Hay quien adquiere la mala costumbre de ser infeliz" (M. A. Evans)

Larry Rosenman wrote:

--On Saturday, June 21, 2003 11:43:17 -0400 Tom Lane
<tgl@sss.pgh.pa.us> wrote:

Peter Eisentraut <peter_e@gmx.net> writes:

Thomas Swan writes:

Have you considered something similar to the Mozilla tinderbox
approach
where you have a daemon checkout the cvs, compile, run regression
tests,
and report a status or be able to report a status?

Even if you could achieve near complete coverage of the platforms,
platform versions, and auxilliary software versions and combinations
that
PostgreSQL runs with, in most cases, something breaks on a new
version or combination of these things.

Still, whenever we're doing something that interacts at all with the OS,
it seems we get breakages that don't show in the original author's
testing, but only pop up days to months later when some beta tester
tries the code on platform P or using option Q. The current
difficulties with the IPv6 patches are a fine case in point.
If we could get feedback more easily about whether a proposed patch
compiles and passes regression on a variety of platforms, we could
reduce the pain involved by a great deal, simply because the problems
could be fixed while the code is still fresh in mind.

I don't think there is any company involved with Postgres that is
willing to commit the resources to run a Mozilla-style tinderbox setup
singlehanded. But I wonder whether we couldn't set up something that is
community-based: get a few dozen people with different platforms to
volunteer to check the code regularly on their own machines. I'm
imagining a cron job that fires daily in the wee hours, pulls the latest
CVS tip, does "make distclean; configure; make; make check", and mails
the results to someplace that puts 'em up on our website.

It's possible that we could adapt the tinderbox software to work this
way, but even if we had to write our own, it seems like a fairly simple
task. And it'd give *much* better feedback on porting problems than we
have now. Sure, there will always be corner cases you don't catch,
but the first rule of testing is the sooner you find a bug the cheaper
it is to fix.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

I'm willing to run such a job on UnixWare 7.1.3 and OpenUnix 8, as well
as FreeBSD 4.8

I'll have a machine shortly where I can run RH9 SMP tests..

Tom Lane wrote:

I have been through crash-me in some detail, and it left a very bad
taste in my mouth. Don't bother holding it up as an example of good
practice.

You seem to miss Dan's point. The specific implementation of crashme
is undoubtedly flawed in a number of ways, but the idea is very useful
as part of an acceptance testing suite. In short, it would probably
be beneficial to us to fix crashme so that it tests the proper,
standards-compliant things and reports the actual results, and then
include it in the test suite.

Indeed, we could even go so far as to use it for our own marketing
purposes! Have it cite, for each test, which part of the SQL spec it's
testing and what the result should be.

--
Kevin Brown kevin@sysexperts.com

I wrote:

Tom Lane wrote:

I have been through crash-me in some detail, and it left a very bad
taste in my mouth. Don't bother holding it up as an example of good
practice.

You seem to miss Dan's point. The specific implementation of crashme
is undoubtedly flawed in a number of ways, but the idea is very useful
as part of an acceptance testing suite. In short, it would probably
be beneficial to us to fix crashme so that it tests the proper,
standards-compliant things and reports the actual results, and then
include it in the test suite.

Actually, now that I think about it, it would probably be more beneficial
to merge any correct tests that we aren't already performing into our
existing regression test framework, provided that the end result doesn't
take too long to run (as you pointed out elsewhere, regression tests
that take a really long time to run simply won't be run by most people,
except perhaps in a tinderbox type of environment).

Overall, it might be of some benefit to mark individual regression tests
with a priority, and then make it possible to run only those tests of
a specified priority or higher. That way, the indvidual developer may
decide for himself which group of regression tests to run based on the
amount of time he's willing to let it take and how much hardware he has
to throw at it. And at the same time, it would make it easier for new
tests to be included in the suite without worrying about the impact it
would have on people running the tests.

--
Kevin Brown kevin@sysexperts.com

----- Original Message -----
From: "Tom Lane" <tgl@sss.pgh.pa.us>
To: "Peter Eisentraut" <peter_e@gmx.net>
Cc: "Thomas Swan" <tswan@idigx.com>; <pgsql-hackers@postgresql.org>
Sent: Saturday, June 21, 2003 11:43 AM
Subject: Re: [HACKERS] Two weeks to feature freeze

I don't think there is any company involved with Postgres that is
willing to commit the resources to run a Mozilla-style tinderbox setup
singlehanded. But I wonder whether we couldn't set up something that is
community-based: get a few dozen people with different platforms to
volunteer to check the code regularly on their own machines. I'm
imagining a cron job that fires daily in the wee hours, pulls the latest
CVS tip, does "make distclean; configure; make; make check", and mails
the results to someplace that puts 'em up on our website.

It's possible that we could adapt the tinderbox software to work this
way, but even if we had to write our own, it seems like a fairly simple
task. And it'd give *much* better feedback on porting problems than we
have now. Sure, there will always be corner cases you don't catch,
but the first rule of testing is the sooner you find a bug the cheaper
it is to fix.

Two thoughts:
1. we'd need a matrix of hardware / (OS/version) / other environmental
things to ensure some sort of good coverage.
2. we'd need to test various configuration sets too, e.g. --with-krb5

I too have an old spare x86 machine lying around that I can set up with
whatever free *nix might not have coverage and contribute to the effort.

andrew

Rod Taylor wrote:

Do we have any "killer" features added to 7.4 that we can shout about?
There's usually been one or two in the past...?

A quick glance at the TODO list shows a number of speed improvements in
specific areas (IN, GROUP BY, Subselects in views), ARRAY improvements,
some utility command improvements / additions, and a significant
protocol update.

The protocol update may not be flashy, but it is a large step forward in
presenting a clean experience for developers using PostgreSQL (reduces
chance of rare, unexpected, and difficult to find logic errors).

If nothing else, it makes for an excellent cleanup release that rounds
off some of the sharp corners (tab completion for schema elements in
psql, schema dump in psql, fixed cluster support, transactional
truncate, alter sequence, new regex code for fast MultiByte, etc).

The problem with cleanup releases is that most of our recent releases
have been of that type. Each release is a good step forward, but I was
hoping for a set of killer features for this release.

Tom said that our low-hanging fruit is gone and only hard items are
left. This is certainly true. What is hard to accept is that those big
items take _weeks_ of focused development, and we just don't have enough
full-time developers who can spend that amount of time to do them. The
sad truth is that there is alway something _else_ to do, rather than
block out weeks to code a complex feature. And these are usually
features that can't be done incrementally, but require a huge input of
time before there is any payback.

I tried with Win32, and spent a few weeks getting us closer, but my
other work of housecleaning (email/patches/cleanup), and marketing
(speaking and tutorial preparation) just make it impossible to spend the
time needed to complete a big item. And people were rightly upset that
the patches weren't getting applied or cleanup done in a timely manner.

It is depressing.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tom Lane wrote:

"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:

What about the nested transaction stuff?

With all due respect to Alvaro et al, I can't imagine that that will
make it into 7.4. (I have no confidence that PITR or Win32 native port
will make it either...)

Do we have any "killer" features added to 7.4 that we can shout about?

We have a lot of pretty good stuff. You're not happy that the
performance of IN (subselect) has been fixed? That btree index bloat is
fixed (at least in large part, it remains to be seen whether the field
performance is all that we need...)?

In my opinion the project is not at a state where whizzy new Features
with a capital F are going to jump out of the woodwork. We are making
good advances in performance, reliability, SQL spec compliance, and
stuff like that, but fancy-sounding bullet points are hard to come by.

What does bother me is that we weren't getting any closer on those
_hard_ items. At least with this release, we will be _closer_ on Win32
and PITR.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Oleg Bartunov wrote:

Do we have any "killer" features added to 7.4 that we can shout about?
There's usually been one or two in the past...?

I'm not sure if contrib/tsearch is a "killer" feature, but we hope
to submit completely new version of tsearch V2 before July 1.
Actually, we have stable code already used in some projects but
currently lacking documentation. Several people are working on tutorial,
reference guide. The problem is that Bruce seems is very overloaded and
for sure he'll have many patches close to July 1. Is it possible
to get rights to commit our changes ?

I am sorry there has been such a delay in patches. I will try go
improve that, or someone else can apply them.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Andrew Dunstan wrote:

Maybe a better strategy would be to get a release out soon but not wait 6
months for another release which would contain the Win32 port and the PITR
stuff (assuming those aren't done in time for this release).

What concerns me is that we thought that after 7.3, and didn't do much
work on either until we got near 7.4 release. I wonder if this is just
going to be a pattern, where these items are so large, we can't get any
motivation to focus on them until we get near the final release. I
guess if each final release gets us a little closer, eventually we will
get there, but this process is not ideal.

The big puzzle is how do you get people (including myself) motivated to
work on a feature that takes a _huge_ amount of work to see any payoff?
I would like to know. Anyone?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Jason Earl wrote:

I'd rather see the dev cycle shortened by a month, then extended ...

Why couldn't you just release the win32 version of 7.4 when it was
finished. If it takes an extra month then that just gives you guys
the chance to circulate *two* press releases. The Native Win32 port
is likely to make a big enough splash all by itself.

I am working to try to get fork/exec and signals in before the feature
freeze so a Win32 patch to 7.4 is possible.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Sat, 21 Jun 2003, Bruce Momjian wrote:

What does bother me is that we weren't getting any closer on those
_hard_ items. At least with this release, we will be _closer_ on Win32
and PITR.

Maybe our problem is such a ... hatred of #ifdef? Maybe its time to go
back a bit to our roots ... get the 'experimental features' in with #ifdef
so that others have a chance to look at and work on it, and once ready for
prime time, pull the #ifdef's out ... ?

Dann Corbit wrote:

Adding a new platform--especially a platform as diverse from
the rest of PostgreSQL's supported platforms as Windows--is
what adds the work. Testing the new platform is relatively
easy. All you need to do is to start using the Win32 version
with real live data.

That is not testing. Using the world as your beta team seems to be a
business model used by a few software giants that is largely frowned
upon. I would think that there is an opportunity to do things
differently. [Read 'properly'].

We (at CONNX Solutions Inc.) have a formal release procedure that
includes many tens of thousands of automated tests using dozens of
different platforms. There are literally dozens of machines (I would
guess 70 or so total) running around the clock for 7 days before we even
know if we have a release candidate. The QA team is distinct from the
development team, and if they say "FLOP!" the release goes nowhere. No
formal release until QA passes it.

If there is no procedure for PostgreSQL of this nature, then there
really needs to be. I am sure that MySQL must have something in place
like that. Their "Crash-Me" test suite shows (at least) that they have
put a large effort into testing.

One thing you might be missing is that we have a _very_ close
relationship with our users. We can send out code and debug/fix things
much faster than a company can that ships binaries. If you look at the
changes that go into minor releases (post X.X.0 releases) you will see
very few fixes, and the ones we do fine are usually for very esoteric
problem cases. Maybe it isn't ideal, but given our limited resources,
it works very well.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Dann Corbit wrote:

Perhaps all that is needed is some sort of automated, formal reporting
procedure. For example, a large test set might be created that runs a
thorough regression feature list. When the test completes, a data file
is emailed to some central repository, parsed, and stored in a database.

I do have an automated build/initdb/regression that I run every night
and email the results to myself.

[ "X$1" != "X-n" ] && CLEAN="-c" && shift

. /etc/profile
pgstop
rm -r /u/pg/data
# return command error value
(pgmakeall $CLEAN 2>&1; echo "$?" > $TMP/ret) |
(tee $TMP/0; exit `cat $TMP/ret`) &&
aspg initdb &&
pgstart &&
newdb test &&
cd /pg/test/regress &&
gmake clean &&
aspg gmake installcheck
grep warning $TMP/0 |
grep -v setproctitle |
grep -v find_rule |
grep -v yy_flex_realloc |
grep -v '\[javac\] 1 warning'

I also run this after I apply patches.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Dann Corbit wrote:

That is the worst possible test plan. It totally lacks organization and
there is no hint to define when the feature set has been covered. Ad
hoc testing is a useful addition, but it cannot replace all the standard
tests that have been used by the industry for decades.

If you run literally hundreds of tests designed to ensure that your
product conforms to ANSI/ISO standards then the bugs that are missed
will be few and far between. Unless you are bad at designing tests.

Designing tests is busywork. Desiging tests is boring. Nobody wants to
design tests, let alone interpret the results and define correct
baselines. But testing is very, very important.

I remember when I was with Great Bridge they said, "Oh, we are going to
have a test setup and do all sorts of testing to improve PostgreSQL." I
told them I doubted their testing was going to shake out many more bugs
than our existing testing setup, and you know what, I was pretty much
right. Sure, they found a few, but it wasn't much.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

The Hermit Hacker wrote:

On Sat, 21 Jun 2003, Bruce Momjian wrote:

What does bother me is that we weren't getting any closer on those
_hard_ items. At least with this release, we will be _closer_ on Win32
and PITR.

Maybe our problem is such a ... hatred of #ifdef? Maybe its time to go
back a bit to our roots ... get the 'experimental features' in with #ifdef
so that others have a chance to look at and work on it, and once ready for
prime time, pull the #ifdef's out ... ?

That's a tough call. I do worry about readability. We have made Win32
changes, and they aren't ifdefs, and we still have a running system, and
I think we can do that for PITR too. I think the big issue, which may be
your point, is to get incremental work into CVS as soon as possible so
we continue to take small steps.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

I have added a cleaned up version of this to CVS as src/tools/pgtest.

---------------------------------------------------------------------------

Bruce Momjian wrote:

Dann Corbit wrote:

Perhaps all that is needed is some sort of automated, formal reporting
procedure. For example, a large test set might be created that runs a
thorough regression feature list. When the test completes, a data file
is emailed to some central repository, parsed, and stored in a database.

I do have an automated build/initdb/regression that I run every night
and email the results to myself.

[ "X$1" != "X-n" ] && CLEAN="-c" && shift

. /etc/profile
pgstop
rm -r /u/pg/data
# return command error value
(pgmakeall $CLEAN 2>&1; echo "$?" > $TMP/ret) |
(tee $TMP/0; exit `cat $TMP/ret`) &&
aspg initdb &&
pgstart &&
newdb test &&
cd /pg/test/regress &&
gmake clean &&
aspg gmake installcheck
grep warning $TMP/0 |
grep -v setproctitle |
grep -v find_rule |
grep -v yy_flex_realloc |
grep -v '\[javac\] 1 warning'

I also run this after I apply patches.

-- 
Bruce Momjian                        |  http://candle.pha.pa.us
pgman@candle.pha.pa.us               |  (610) 359-1001
+  If your life is a hard drive,     |  13 Roberts Road
+  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom said that our low-hanging fruit is gone and only hard items are
left. This is certainly true. What is hard to accept is that those big
items take _weeks_ of focused development, and we just don't have enough
full-time developers who can spend that amount of time to do them. The
sad truth is that there is alway something _else_ to do, rather than
block out weeks to code a complex feature. And these are usually
features that can't be done incrementally, but require a huge input of
time before there is any payback.

I spent weeks doing hash aggregates, weeks doing IN-subselect
optimization, and am in the middle of many weeks on FE/BE protocol
improvement. I am sorry that you don't see these as killer features
... but they are all things that we desperately needed to do.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom said that our low-hanging fruit is gone and only hard items are
left. This is certainly true. What is hard to accept is that those big
items take _weeks_ of focused development, and we just don't have enough
full-time developers who can spend that amount of time to do them. The
sad truth is that there is alway something _else_ to do, rather than
block out weeks to code a complex feature. And these are usually
features that can't be done incrementally, but require a huge input of
time before there is any payback.

I spent weeks doing hash aggregates, weeks doing IN-subselect
optimization, and am in the middle of many weeks on FE/BE protocol
improvement. I am sorry that you don't see these as killer features
... but they are all things that we desperately needed to do.

Yes, I know they are _very_ needed, but they don't increase
functionality the way Win32 or PITR would do.

Please don't feel I am minimizing these features. If I had to choose, I
would choose those features over Win32 or PITR. It is just that I
wanted all of them. :-(

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

----- Original Message -----
From: "Bruce Momjian" <pgman@candle.pha.pa.us>

Tom said that our low-hanging fruit is gone and only hard

items are

left. This is certainly true. What is hard to accept is that

those big

items take _weeks_ of focused development, and we just don't

have enough

full-time developers who can spend that amount of time to do

them. The

sad truth is that there is alway something _else_ to do,

rather than

block out weeks to code a complex feature. And these are

usually

features that can't be done incrementally, but require a huge

input of

time before there is any payback.

I tried with Win32, and spent a few weeks getting us closer,

but my

other work of housecleaning (email/patches/cleanup), and

marketing

(speaking and tutorial preparation) just make it impossible to

spend the

time needed to complete a big item. And people were rightly

upset that

the patches weren't getting applied or cleanup done in a

timely manner.

It is depressing.

I was disappointed that Satoshi Nagayasu's two-phase commit
patches seemed to be implicitly rejected by lack of an
enthusiastic response by any of the core members. Distributed
query (not replication) would have been a very nice feature.
It's what separates, in part, Oracle Enterprise Edition from the
Standard Edition, and it appeared someone (Satoshi Nagayasu) was
more than willing to get the ball rolling. But the flight path
bothered some I guess so we got nothin'

Mike Mascari
mascarm@mascari.com

Tom Lane wrote:

I spent weeks doing hash aggregates, weeks doing IN-subselect
optimization, and am in the middle of many weeks on FE/BE protocol
improvement. I am sorry that you don't see these as killer features
... but they are all things that we desperately needed to do.

For me, the 7.4 enhancements are essential, the join optimizations make
the difference between "app works" and "app doesn't work", because
queries (that can't be changed) that previously ran for ages for
non-obvious reasons now speed up to <<1 second. The planner reached a
new level of maturity.

I'd recommend continuing enhancement work on pgsql, and if a majority
feels that features are so killing the version is bumped up one major.
I wouldn't see a yet another platform as a reason for this, rather
something that vastly extends the field of operation (2PC was mentioned,
maybe PITR).

Regards,
Andreas

Mike Mascari wrote:

I was disappointed that Satoshi Nagayasu's two-phase commit
patches seemed to be implicitly rejected by lack of an
enthusiastic response by any of the core members. Distributed
query (not replication) would have been a very nice feature.
It's what separates, in part, Oracle Enterprise Edition from the
Standard Edition, and it appeared someone (Satoshi Nagayasu) was
more than willing to get the ball rolling. But the flight path
bothered some I guess so we got nothin'

I sure want two-phase commit. I don't remember it as being rejected,
and we certainly need it, independent of replication.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tom Lane wrote:

BTW, I would not approve of a response along the lines of "can't you
#ifdef to the point that there are no code changes in the Unix builds?"
No you can't, unless you want to end up with an unmaintainable mess
of #ifdef spaghetti. The thing that makes this hard is the tradeoff
between making the code readable and maintainable (which requires
sharing as much code as possible across platforms) vs isolating
platform-specific considerations. Programming at this level is not
a science but an art form, and it's very hard to get it right the first
time --- especially when none of us have access to all the platforms
that the code must ultimately work on.

Exactly my point and the reason I am doing the entire fork+exec stuff
over again. Bruce nagged me endlessly to commit the broken parts I had
and fix them later. I never agreed with that philosophy because in my
experience the worst workarounds live forever.

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #

Dann Corbit wrote:

PostgreSQL's regression tests (IMHO) are much better than
MySQL's crash-me scripts.

They are less thorough in coverage, but not too bad.

Right, we are still missing this test that proves 10,000 CREATE+DROP
TABLE will ensure that PostgreSQL is slower than MySQL, if you don't
VACUUM the catalog ...

Here is what I suggest:

PostgreSQL has an excellent programming team. Why not try to recruit a
similar testing team? I think it would strongly differentiate the tool
set from similar free stuff.

Perhaps all that is needed is some sort of automated, formal reporting
procedure. For example, a large test set might be created that runs a
thorough regression feature list. When the test completes, a data file
is emailed to some central repository, parsed, and stored in a database.

Here is what I suggest:

Get started :-)

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #

Dann Corbit wrote:

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]

Are you volunteering to create it? Step right up.

No. And as an outsider, I rather doubt if any procedures I developed
would be taken very seriously. If such procedures are to be developed,
I suspect that they will have to be developed from within if they are to
be successful.

What do you think is the way to become an insider?

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #

Alvaro Herrera wrote:

Also they don't test things they don't support. Is there a test for
subselects? What about concurrency? Transactional issues? What about
performance when they have their "transaction support" enabled?

Sure don't they. Like their NUMERIC data type, that can *store* any
precision, but when you actually calculate with it, it converts to
floating point internally ... that's not only a spec violation, in many
countries that's a violation of law if used by a bookkeeping system.

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #

On Sun, 2003-06-22 at 07:44, Bruce Momjian wrote:

Mike Mascari wrote:

I was disappointed that Satoshi Nagayasu's two-phase commit
patches seemed to be implicitly rejected by lack of an
enthusiastic response by any of the core members. Distributed

They weren't ready to be committed at the time, nor are they now.

The hardest parts are still to come (resume, forget, etc.).

I believe he is still working on the third phase:

http://snaga.org/pgsql/

--
Rod Taylor <rbt@rbt.ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

Jan Wieck wrote:

Tom Lane wrote:

BTW, I would not approve of a response along the lines of "can't you
#ifdef to the point that there are no code changes in the Unix builds?"
No you can't, unless you want to end up with an unmaintainable mess
of #ifdef spaghetti. The thing that makes this hard is the tradeoff
between making the code readable and maintainable (which requires
sharing as much code as possible across platforms) vs isolating
platform-specific considerations. Programming at this level is not
a science but an art form, and it's very hard to get it right the first
time --- especially when none of us have access to all the platforms
that the code must ultimately work on.

Exactly my point and the reason I am doing the entire fork+exec stuff
over again. Bruce nagged me endlessly to commit the broken parts I had
and fix them later. I never agreed with that philosophy because in my
experience the worst workarounds live forever.

I wouldn't say nagging ... I would say NAGGING. :-)

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Sat, 21 Jun 2003, Bruce Momjian wrote:

That's a tough call. I do worry about readability. We have made Win32
changes, and they aren't ifdefs, and we still have a running system, and
I think we can do that for PITR too. I think the big issue, which may be
your point, is to get incremental work into CVS as soon as possible so
we continue to take small steps.

Exactly ... its gotten to the point that the changes needed are large, so
ppl are waiting until its all finished before submitting it ...

On Sun, 22 Jun 2003, Bruce Momjian wrote:

Tom Lane wrote:

I spent weeks doing hash aggregates, weeks doing IN-subselect
optimization, and am in the middle of many weeks on FE/BE protocol
improvement. I am sorry that you don't see these as killer features
... but they are all things that we desperately needed to do.

Yes, I know they are _very_ needed, but they don't increase
functionality the way Win32 or PITR would do.

They don't increase functionality for whom? When someone is comparing
PostgreSQL to Oracle, as an example, for consideration in a project, I
would think that speed would be one thing that they would consider key
'functionality' in that comparison ... no?

I'll never use a Win32 port ... so Tom's work on optimizing queries is
more important to me then a Win32 port is ... 'functionality' is
completely in 'the eye of the beholder' ...

On Sun, 22 Jun 2003, Bruce Momjian wrote:

Mike Mascari wrote:

I was disappointed that Satoshi Nagayasu's two-phase commit
patches seemed to be implicitly rejected by lack of an
enthusiastic response by any of the core members. Distributed
query (not replication) would have been a very nice feature.
It's what separates, in part, Oracle Enterprise Edition from the
Standard Edition, and it appeared someone (Satoshi Nagayasu) was
more than willing to get the ball rolling. But the flight path
bothered some I guess so we got nothin'

I sure want two-phase commit. I don't remember it as being rejected,
and we certainly need it, independent of replication.

I don't recall the patch itself :(

Mike, do you recall the date(s) for this? Reasons for rejections?

On Fri, 20 Jun 2003, Dann Corbit wrote:

Designing tests is busywork. Desiging tests is boring. Nobody wants to
design tests, let alone interpret the results and define correct
baselines. But testing is very, very important.

But we do do testing ... we even design testing (in the form of the
regression tests) ... we just don't do testing that you personally approve
of ... and, from what I've seen so far, you aren't willing to actually put
*your* time where your mouth is ... design some tests and submit them to
us ... if they are valid, they will get used ...

If you feel that crash-me is a valid starting point, start there and see
where it takes you ...

On Sat, 21 Jun 2003, Dann Corbit wrote:

It seems pretty clear that there are warts on the Crashme test.
Perhaps 70% or so is truly useful. Maybe the useful subset could be
approximated or modified to be useful as a general tool set.

And we all wait with baited breath for you to develop and submit such
tests ...

The Hermit Hacker wrote:

On Sun, 22 Jun 2003, Bruce Momjian wrote:

Mike Mascari wrote:

I was disappointed that Satoshi Nagayasu's two-phase commit
patches seemed to be implicitly rejected by lack of an
enthusiastic response by any of the core members. Distributed
query (not replication) would have been a very nice feature.
It's what separates, in part, Oracle Enterprise Edition from the
Standard Edition, and it appeared someone (Satoshi Nagayasu) was
more than willing to get the ball rolling. But the flight path
bothered some I guess so we got nothin'

I sure want two-phase commit. I don't remember it as being rejected,
and we certainly need it, independent of replication.

I don't recall the patch itself :(

Mike, do you recall the date(s) for this? Reasons for rejections?

I choose my words poorly. A discussion arose regarding the 7.4
protocol changes. I suggested looking forward to allow for a 2PC
implementation. Satoshi Nagayasu remarked about the work done on 2PC
and posted a link to patches:

http://snaga.org/pgsql/pgsql-20021025.tgz

The thread was here:

http://archives.postgresql.org/pgsql-hackers/2002-11/msg00143.php

Various people critiqued the work that had been done - protocol change
instead of a purely statement-driven implementation, the use of 2PC
for sync. replication, etc. And that was the last (and first, IIRC)
post from Satoshi Nagayasu. I was worried that PostgreSQL lost the
opportunity to have a 2PC implementation, because no one followed up,
allowing it to die on the vine.

I have learned from Rod Taylor that lack of posts on hackers doesn't
mean lack of work:

"They weren't ready to be committed at the time, nor are they now.
The hardest parts are still to come (resume, forget, etc.).
I believe he is still working on the third phase:

http://snaga.org/pgsql/

-- Rod Taylor <rbt@rbt.ca>"

So I stand corrected.

Mike Mascari
mascarm@mascari.com

The Hermit Hacker wrote:

On Fri, 20 Jun 2003, Dann Corbit wrote:

Designing tests is busywork. Desiging tests is boring. Nobody wants to
design tests, let alone interpret the results and define correct
baselines. But testing is very, very important.

But we do do testing ... we even design testing (in the form of the
regression tests) ... we just don't do testing that you personally approve
of ... and, from what I've seen so far, you aren't willing to actually put
*your* time where your mouth is ... design some tests and submit them to
us ... if they are valid, they will get used ...

If you feel that crash-me is a valid starting point, start there and see
where it takes you ...

Not that fast! I didn't take the time to check but it wouldn't surprise
me if MySQL's crash-me is GPL'd and copyright MySQL AB. That's not an
optimal point to start PostgreSQL test code from, is it?

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #

On Sun, 22 Jun 2003, Jan Wieck wrote:

The Hermit Hacker wrote:

On Fri, 20 Jun 2003, Dann Corbit wrote:

Designing tests is busywork. Desiging tests is boring. Nobody wants to
design tests, let alone interpret the results and define correct
baselines. But testing is very, very important.

But we do do testing ... we even design testing (in the form of the
regression tests) ... we just don't do testing that you personally approve
of ... and, from what I've seen so far, you aren't willing to actually put
*your* time where your mouth is ... design some tests and submit them to
us ... if they are valid, they will get used ...

If you feel that crash-me is a valid starting point, start there and see
where it takes you ...

Not that fast! I didn't take the time to check but it wouldn't surprise
me if MySQL's crash-me is GPL'd and copyright MySQL AB. That's not an
optimal point to start PostgreSQL test code from, is it?

I didn't say to copy it, but if the format is what Dann feels is required
to be taken seriously, it does give a starting point to work from ...

the thing is, as I think it was Tom that pointed out, the crash-me is more
a feature tester then anything ... but Dann appears to be stuck on it as
the 'be all, end all of testing suites' ...

Bruce Momjian writes:

I have added a cleaned up version of this to CVS as src/tools/pgtest.

This seems to be a platform-specific reimplementation of 'make clean; make
check'. Why bother?

--
Peter Eisentraut peter_e@gmx.net

The Hermit Hacker wrote:

And, actually, for some reason I hadn't thought of the tsearch as being
another 'INDEX' type ... I crawl back over and be quiet now :)

Oleg, as far as commits are concerned, I have no problems with extending
the privileges to one of your guys for this, just email me seperately who,
and I'll get it setup ...

That would help me too.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Peter Eisentraut wrote:

Bruce Momjian writes:

I have added a cleaned up version of this to CVS as src/tools/pgtest.

This seems to be a platform-specific reimplementation of 'make clean; make
check'. Why bother?

Marc requested it. Is there anything platform-specific except the greps?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Let me add my two cents ..

I think something like PostgreSQL needs two test suites - an
acceptance test (to ensure that checkins don't break functionality)
and a regression test suite.

What we call the regression test suite is really an acceptance
test. Tom Lane is absolutely right in asserting that a test suite that
takes a week to run will mean that people won't test at
all. Personally, I can (and have for many years) tolerate acceptance
test suites that take upto an hour.

The existence of such an acceptance test should not however obviate
the presence of a wider regression test suite. It should be fine to
have an entire suite of regression test buckets take a week to run,
because you only start running them once you have a release candidate
or equivalent.

Of course, setting up a regression test suite takes effort. There is
no need, however, to spend umpteen amounts of time writing the
buckets. What can be done is to incrementally build it up. So whenever
we have a significant new feature, somebody (preferably not the key
developers) could take the time to set up a set of test cases that try
to test it thoroughly. It's okay if such a test bucket takes 10-15
minutes to run. Then this can get rolled up into the regression suite
while a very small "representative" test case makes it to the
acceptance test suite.

Of course, in the open source world, these things take resources and
are not easy to do.

I certainly think that the base regression test suite is great. We
have clearing the pgsql regression test a checkin requirement for
TelegraphCQ developers as our goal is to not break pgsql
functionality.

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

On Sun, 22 Jun 2003, Bruce Momjian wrote:

Peter Eisentraut wrote:

Bruce Momjian writes:

I have added a cleaned up version of this to CVS as src/tools/pgtest.

This seems to be a platform-specific reimplementation of 'make clean; make
check'. Why bother?

Marc requested it. Is there anything platform-specific except the greps?

Ya, the script looked like it did a bit more then just a 'make clean; make
check' ... doesn't it?

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I sure want two-phase commit. I don't remember it as being rejected,
and we certainly need it, independent of replication.

Is 2PC a real-world solution to any real-world problem? I have never
seen a satisfactory explanation of what you do when you've reported that
you're ready to commit and no confirmation ever comes back. Sooner or
later you must violate the protocol in one direction or the other (ie,
commit without confirmation or roll back in violation of your promise
of being able to commit).

I think it's a cool-sounding phrase that does not actually work in
practice.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I sure want two-phase commit. I don't remember it as being rejected,
and we certainly need it, independent of replication.

Is 2PC a real-world solution to any real-world problem? I have never
seen a satisfactory explanation of what you do when you've reported that
you're ready to commit and no confirmation ever comes back. Sooner or
later you must violate the protocol in one direction or the other (ie,
commit without confirmation or roll back in violation of your promise
of being able to commit).

I think it's a cool-sounding phrase that does not actually work in
practice.

I think 2PC can be used to build more complex features, like using dblink
for cross-db modification queries.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Sun, 22 Jun 2003, Bruce Momjian wrote:

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I sure want two-phase commit. I don't remember it as being rejected,
and we certainly need it, independent of replication.

Is 2PC a real-world solution to any real-world problem? I have never
seen a satisfactory explanation of what you do when you've reported that
you're ready to commit and no confirmation ever comes back. Sooner or
later you must violate the protocol in one direction or the other (ie,
commit without confirmation or roll back in violation of your promise
of being able to commit).

I think it's a cool-sounding phrase that does not actually work in
practice.

I think 2PC can be used to build more complex features, like using
dblink for cross-db modification queries.

That was my understanding too ... as soon as you try and go distributed,
you need some method of ensuring that the data is constant across them all
...

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

I think it's a cool-sounding phrase that does not actually work in
practice.

I think 2PC can be used to build more complex features,

Only if it works to begin with. If it's unreliable, what are you gonna
build on it?

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

I think it's a cool-sounding phrase that does not actually work in
practice.

I think 2PC can be used to build more complex features,

Only if it works to begin with. If it's unreliable, what are you gonna
build on it?

The question was whether 2PC is useful. The question wasn't if an
unreliable 2PC was useful.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

The question was whether 2PC is useful. The question wasn't if an
unreliable 2PC was useful.

My question is whether there is such a thing as reliable 2PC. I sure
don't see how you build that.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

The question was whether 2PC is useful. The question wasn't if an
unreliable 2PC was useful.

My question is whether there is such a thing as reliable 2PC. I sure
don't see how you build that.

Other databases use 2PC --- are you saying none of them are reliable?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

I sure want two-phase commit. I don't remember it as being rejected,
and we certainly need it, independent of replication.

Is 2PC a real-world solution to any real-world problem? I have never
seen a satisfactory explanation of what you do when you've reported that
you're ready to commit and no confirmation ever comes back. Sooner or
later you must violate the protocol in one direction or the other (ie,
commit without confirmation or roll back in violation of your promise
of being able to commit).

I think it's a cool-sounding phrase that does not actually work in
practice.

The other problem I was missing being addressed is what happens if one
promised "I can commit" and crashes? Not exactly at the time he crashes,
but more at the time he restarts? Doesn't he have to restart into
exactly that state of "I can commit", with all locks in place and yet
being able to rollback and then again ask "and what now"? I would be
surprised if said patch does that ... very *positively* surprised!

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Other databases use 2PC --- are you saying none of them are reliable?

Perhaps they're just smarter than I am. My question stands: what do
you do when the controller doesn't respond after you promise to commit?
Without a believable answer to that, I have no confidence in the idea.

regards, tom lane

"Bruce" == Bruce Momjian <pgman@candle.pha.pa.us> writes:

Bruce> Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes: > The question
was whether 2PC is useful. The question wasn't if an >
unreliable 2PC was useful.

My question is whether there is such a thing as reliable 2PC.
I sure don't see how you build that.

Bruce> Other databases use 2PC --- are you saying none of them are
Bruce> reliable?

And they use them for both federated read/write (what you refer to as
distributed access through dblink) and for clustered configurations.

I'm not sure if I understand Tom's beef - I think he is concerned
about what happens if a subordinate does not respond to a prepare
message. I would assume that the co-ordinator would not let the commit
go through until it has received confirmations from every
subordinate. The application's commit will remain blocked against the
co-ordinator when this takes place.

That said, I agree that 2PC (and variants) is rather heavy weight when
in widely distributed configurations.

(Although I guess in practice, many people use Presumed Abort and not
vanilla 2PC as PA results in fewer log flushes for read-only
transactions.)

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

Jan Wieck <JanWieck@Yahoo.com> writes:

The other problem I was missing being addressed is what happens if one
promised "I can commit" and crashes? Not exactly at the time he crashes,
but more at the time he restarts? Doesn't he have to restart into
exactly that state of "I can commit", with all locks in place

Yes, I think he does --- which adds a whole 'nother layer of complexity
and performance penalty to the thing, because all those held locks etc
have to be recorded on disk before you promise to commit.

That part is soluble in theory though, ie, I believe that it can be
done (not efficiently, but it can be done). I don't see what to do
about the no-commit-ack problem.

regards, tom lane

Sailesh Krishnamurthy <sailesh@cs.berkeley.edu> writes:

I'm not sure if I understand Tom's beef - I think he is concerned
about what happens if a subordinate does not respond to a prepare
message. I would assume that the co-ordinator would not let the commit
go through until it has received confirmations from every
subordinate.

No. I want to know what the subordinate does when it's promised to
commit and the co-ordinator never responds. AFAICS the subordinate
is screwed --- it can't commit, and it can't abort, and it can't expect
to make progress indefinitely on other work while it's holding locks
for the not-quite-committed transaction.

regards, tom lane

No. I want to know what the subordinate does when it's promised to
commit and the co-ordinator never responds. AFAICS the subordinate
is screwed --- it can't commit, and it can't abort, and it can't expect
to make progress indefinitely on other work while it's holding locks
for the not-quite-committed transaction.

It takes itself offline and you need to resync it later??

Chris

On Sun, 22 Jun 2003, Bruce Momjian wrote:

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

I think it's a cool-sounding phrase that does not actually work in
practice.

I think 2PC can be used to build more complex features,

Only if it works to begin with. If it's unreliable, what are you gonna
build on it?

The question was whether 2PC is useful. The question wasn't if an
unreliable 2PC was useful.

I have to back Bruce up on this one ... is there a reason why 2PC couldn't
be made reliable? I'm guessin that Oracle supports 2PC ... ? If so, is
it unreliable?

On Mon, 23 Jun 2003, Sailesh Krishnamurthy wrote:

I'm not sure if I understand Tom's beef - I think he is concerned about
what happens if a subordinate does not respond to a prepare message. I
would assume that the co-ordinator would not let the commit go through
until it has received confirmations from every subordinate. The
application's commit will remain blocked against the co-ordinator when
this takes place.

Wouldn't 2PC have some sort of 'heartbeat' between the co-ordinator and
subordinate? Like, if you had multiple subordinates and one crashed, the
co-ordinator would have to know that and be able to continue on, no?

On Mon, 23 Jun 2003, Christopher Kings-Lynne wrote:

No. I want to know what the subordinate does when it's promised to
commit and the co-ordinator never responds. AFAICS the subordinate
is screwed --- it can't commit, and it can't abort, and it can't expect
to make progress indefinitely on other work while it's holding locks
for the not-quite-committed transaction.

It takes itself offline and you need to resync it later??

Hrmmm, I see Tom's point (I think!) ... but what if, for instance, the
co-ordinator crashes? From the subordinates point of view, it has the
complete transaction to commit, but the co-ordinator has gone down without
telling it to do so ...

The Hermit Hacker <scrappy@postgresql.org> writes:

Hrmmm, I see Tom's point (I think!) ... but what if, for instance, the
co-ordinator crashes?

Or you just lose the network connection for awhile. The worst case
scenario I think is where the co-ordinator got everyone's promise to
commit, and told some of the subordinates to commit, but your own
response gets lost due to network failure. Now what? If you time
out and decide to abort, you're inconsistent with the other
subordinates. On the other hand, you can't commit after a timeout
either, because that loses in the other scenario (where the coordinator
didn't decide to commit). Basically, the subordinate must be willing
to hold its breath *forever*. I don't see how this can work.

regards, tom lane

"Tom" == Tom Lane <tgl@sss.pgh.pa.us> writes:

Tom> Sailesh Krishnamurthy <sailesh@cs.berkeley.edu> writes:

I'm not sure if I understand Tom's beef - I think he is
concerned about what happens if a subordinate does not respond
to a prepare message. I would assume that the co-ordinator
would not let the commit go through until it has received
confirmations from every subordinate.

Tom> No. I want to know what the subordinate does when it's
Tom> promised to commit and the co-ordinator never responds.
Tom> AFAICS the subordinate is screwed --- it can't commit, and it
Tom> can't abort, and it can't expect to make progress
Tom> indefinitely on other work while it's holding locks for the
Tom> not-quite-committed transaction.

Okay I understand what you mean now.

AFAIK the general way things happen is that each site has a "recovery
procedure" that kicks in after a crash. If the co-ordinator crashes
(which could be before or after it sends out COMMIT messages to some
of the subordinates), its recovery manager will bring the system up,
read the log and ready information about all uncommitted transactions
in virtual storage.

If a Xact is in the PREPARE stage it will periodically send a message
to the co-ordinator asking about what happened to the transaction in
question. Once the co-ordinator has come back online it can respond to
the query.

Of course in the case of a co-ordinator going out of action totally
and remaining unconnected this is not a viable solution.

If you're making the case that 2PC is not viable on very wide area
networks with intermitted connectivity, I agree whole-heartedly.

That said, 2PC (and its children, PA and PC) have their place, and are
indeed used in many systems.

For instance, say you are rigging up a message queueing infrastructure
(like MQ-series) to your database (say with NOTIFY), you'd at least
like to have the db act as a co-ordinator with the MQ.

Or the parallel cluster example I gave earlier. Clustered linux boxes
are definitely here although no open-source DBMS offers a parallel
solution.

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

Tom Lane wrote:

The Hermit Hacker <scrappy@postgresql.org> writes:

Hrmmm, I see Tom's point (I think!) ... but what if, for instance, the
co-ordinator crashes?

Or you just lose the network connection for awhile. The worst case
scenario I think is where the co-ordinator got everyone's promise to
commit, and told some of the subordinates to commit, but your own
response gets lost due to network failure. Now what? If you time
out and decide to abort, you're inconsistent with the other
subordinates. On the other hand, you can't commit after a timeout
either, because that loses in the other scenario (where the coordinator
didn't decide to commit). Basically, the subordinate must be willing
to hold its breath *forever*.

Yep. And if the cohort crashes while waiting for the coordinator to
come back on-line, if I understand the world correctly, it must be
capable of committing the database changes associated with the
COMMIT-VOTE response it supplied to the coordinator's PREPARE. It
seems this would require REDO? And yet there are thousands of
installed distributed databases running enterprises every day.

A paper on a "A New Presumed Commit Optimization for Two Phase Commit"
describes the cohort as:

"If a prepared cohort does not receive a transaction outcome message
promptly, or crashes without remembering the outcome, the cohort asks
the coordinator for the outcome. It keeps on asking until it gets an
answer. (This is the blocking aspect of 2PC.)"

I'd just like to point out that:

1) The XA interface defines a 2PC protocol library which allows
transaction managers, such as BEAS Tuxedo (and Oracle, for that
matter) to use the database in a distributed transaction. Lack of an
XA interface for PostgreSQL prohibits its use in major enterprise
applications. BEAS Tuxedo can talk to PostgreSQL, but won't allow it
to participate in a distributed tx.

2) The users of distributed databases will/should/can know that a
cohort will block waiting for the coordinator. We're not talking
asynchronous multi-master replication of 4 databases distributed over
low-speed communication lines across the country. We're talking about
the sales dept. database having a few linked tables to the accounting
dept. database, where inserts into the one result in inserts into the
other.

Mike Mascari
mascarm@mascari.com

I wrote:

Tom Lane wrote:

Basically, the subordinate must be willing to hold its breath *forever*.

Yep. And if the cohort crashes while waiting for the coordinator to
come back on-line, if I understand the world correctly, it must be
capable of committing the database changes associated with the
COMMIT-VOTE response it supplied to the coordinator's PREPARE. It
seems this would require REDO? And yet there are thousands of
installed distributed databases running enterprises every day.

Please ignore the REDO remark. It's late where I am...

Mike Mascari
mascarm@mascari.com

Scott MArlowe wrote:

On Sat, 21 Jun 2003, Bruce Momjian wrote:

The big puzzle is how do you get people (including myself) motivated
to work on a feature that takes a _huge_ amount of work to see any
payoff? I would like to know. Anyone?

Pizza? :-)

Unfortunately it's off my diet :-(

Seriously, I think an increased willingness to share the work around a bit
would be beneficial. I know that I volunteered to work on the w32 port at a
time when I was, as they say, "between jobs". The response was not
encouraging. Now I am working again and don't have much time available.

I know you can't simply divide tasks with infinite granularity ("nine women
can't make a bay in a month"). Some tasks require one or a few people to get
done and that's all that can be done. But if she/he/they can't get it done,
it's time to send out a call for help, IMNSHO.

Not meaning to criticize - the core team does a great job. I, too, have a
tendency to overcommit and under-delegate. It's very understandable.

andrew

On Sat, 21 Jun 2003, Christopher Kings-Lynne wrote:

Crash-me has nothing to do with testing, it jsut checks to see what
features a db supports:

An interesting point is that until recently, crashme said that the
postgresql backend crashed on very large queries. The actual problem was
that postgresql has NO LIMIT to query size, and the crashme script would
keep feeding the postgresql backend larger and larger amounts of query
until the internal buffer of the crashme script overran.

This failure was attributed to postgresql when it was, in fact a bug in
the crashme script.

This is not an isolated behaviour of crashme. It's a quick dirty hack job
designed to show the differences between MySQL and all the other
databases. If it was truly comprehensive (i.e. SQL92 spec testing) there
would be hundreds of failure points for MySQL. but it isn't. It tests
only those things that are good in MySQL against other databases (for the
most part, there is some token effort at including a few things MySQL
doesn't do).

On Sat, 21 Jun 2003, Bruce Momjian wrote:

Andrew Dunstan wrote:

Maybe a better strategy would be to get a release out soon but not wait 6
months for another release which would contain the Win32 port and the PITR
stuff (assuming those aren't done in time for this release).

What concerns me is that we thought that after 7.3, and didn't do much
work on either until we got near 7.4 release. I wonder if this is just
going to be a pattern, where these items are so large, we can't get any
motivation to focus on them until we get near the final release. I
guess if each final release gets us a little closer, eventually we will
get there, but this process is not ideal.

The big puzzle is how do you get people (including myself) motivated to
work on a feature that takes a _huge_ amount of work to see any payoff?
I would like to know. Anyone?

Pizza? :-)

The Hermit Hacker writes:

Ya, the script looked like it did a bit more then just a 'make clean; make
check' ... doesn't it?

No.

--
Peter Eisentraut peter_e@gmx.net

On Sat, 2003-06-21 at 22:55, Bruce Momjian wrote:

Andrew Dunstan wrote:

Maybe a better strategy would be to get a release out soon but not wait 6
months for another release which would contain the Win32 port and the PITR
stuff (assuming those aren't done in time for this release).

What concerns me is that we thought that after 7.3, and didn't do much
work on either until we got near 7.4 release. I wonder if this is just
going to be a pattern, where these items are so large, we can't get any
motivation to focus on them until we get near the final release. I
guess if each final release gets us a little closer, eventually we will
get there, but this process is not ideal.

The big puzzle is how do you get people (including myself) motivated to
work on a feature that takes a _huge_ amount of work to see any payoff?
I would like to know. Anyone?

Here's a sure to be wildly unpopular suggestion:

Make the deciding factor for the next release support of "foo" (foo can
be win32, pitr, replication, 2PC, whatever...). This would put ample
pressure on the developers of "foo" to get it done since the whole
release rides on their shoulders. It also might encourage others to help
out more since they can't get something new until "foo" is completed.
This would also prioritize said developers time on the large project
rather than other non-prioritized tasks, and it also helps to ensure a
"killer feature" for those who want such things,

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat <xzilla@users.sourceforge.net> writes:

Here's a sure to be wildly unpopular suggestion:

Make the deciding factor for the next release support of "foo" (foo can
be win32, pitr, replication, 2PC, whatever...).

We've done that before (see WAL in 7.1), with unhappy results. The
fundamental problem with it is that towards the end of the cycle,
other developers don't know how to schedule their time, because they
don't know when feature freeze is really going to be. People end up
twiddling their thumbs while the schedule slips a few days at a time.

The target-date-based approach we've taken in the last couple of
releases seems much more productive.

regards, tom lane

On Monday 23 June 2003 10:43 am, Tom Lane wrote:

Robert Treat <xzilla@users.sourceforge.net> writes:

Here's a sure to be wildly unpopular suggestion:

Make the deciding factor for the next release support of "foo" (foo can
be win32, pitr, replication, 2PC, whatever...).

We've done that before (see WAL in 7.1), with unhappy results.

well, I did say it would be wildly unpopular ;-)

The
fundamental problem with it is that towards the end of the cycle,
other developers don't know how to schedule their time, because they
don't know when feature freeze is really going to be. People end up
twiddling their thumbs while the schedule slips a few days at a time.

Ok, what if feature freeze comes 1 month after completion of "foo" feature.
This way the release is still feature dependent, but people arn't sitting
around day by day waiting for foo, and they also don't have to worry about
getting caught in the middle of something when foo gets done. (i'm kind of
picking 1 month arbitraily, this could be two weeks if that works better).

The target-date-based approach we've taken in the last couple of
releases seems much more productive.

productive on a small scale; for sure. productive for large scale features...
well, that's why it's being discussed.

Robert Treat

Peter Eisentraut wrote:

The Hermit Hacker writes:

Ya, the script looked like it did a bit more then just a 'make clean; make
check' ... doesn't it?

No.

Well, it is a nice test template for people who aren't shell script
experts, and I have been in the habit of pushing stuff I use into /tools
so it is available for others.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Mon, 23 Jun 2003, Bruce Momjian wrote:

Peter Eisentraut wrote:

The Hermit Hacker writes:

Ya, the script looked like it did a bit more then just a 'make clean; make
check' ... doesn't it?

No.

Well, it is a nice test template for people who aren't shell script
experts, and I have been in the habit of pushing stuff I use into /tools
so it is available for others.

Please keep pushing such scripts out. They're a valuable learning tool
for many beginners and a cost little in size to be included.

-----Original Message-----
From: The Hermit Hacker [mailto:scrappy@postgresql.org]
Sent: Sunday, June 22, 2003 12:30 PM
To: Jan Wieck
Cc: The Hermit Hacker; Dann Corbit; Tom Lane; Jason Earl;
PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

On Sun, 22 Jun 2003, Jan Wieck wrote:

The Hermit Hacker wrote:

On Fri, 20 Jun 2003, Dann Corbit wrote:

Designing tests is busywork. Desiging tests is boring. Nobody
wants to design tests, let alone interpret the results

and define

correct baselines. But testing is very, very important.

But we do do testing ... we even design testing (in the

form of the

regression tests) ... we just don't do testing that you

personally

approve of ... and, from what I've seen so far, you

aren't willing

to actually put
*your* time where your mouth is ... design some tests and

submit them to

us ... if they are valid, they will get used ...

If you feel that crash-me is a valid starting point,

start there and

see where it takes you ...

Not that fast! I didn't take the time to check but it wouldn't
surprise me if MySQL's crash-me is GPL'd and copyright MySQL AB.
That's not an optimal point to start PostgreSQL test code

from, is it?

I didn't say to copy it, but if the format is what Dann feels
is required to be taken seriously, it does give a starting
point to work from ...

the thing is, as I think it was Tom that pointed out, the
crash-me is more a feature tester then anything ... but Dann
appears to be stuck on it as the 'be all, end all of testing
suites' ...

No. I think it covers a broad spectrum of functionality. It is clear
that there are warts in it, and also that it is slanted in a few
instances to turn "bugs into features." But I think that a large and
thorough test suite that covers all major areas of functionality will
prove useful. A test suite that covers just as many features and yet is
aimed at honest evaluation would be a big benefit.

The larger and more complete a functionality test suite is, the better.
If a test suite covers ten times the functionality, it will uncover ten
times as many defects. I think it is part of the responsibility of a
software vendor to ensure that any released product is as free of
defects as possible (even an open source tool set). All real software
products larger than a few hundred lines of code have some defects in
them. If you are going to trust your companies data to a software tool,
you would want it to be as free from defects as is possible to achieve,
without rasing the cost prohibitively.

I think that performance testing is also a good idea. One of the big
benefits of creating a large performance suite is that you can profile
your code and find out where the effort is needed to get a big speed
increase.

I think that the NIST validation suite will be very valuable. The
coverage of NIST is pretty good, but that test has warts on it too. You
will find (for instance) that there is not one single index built by
that test suite. So the joins are all brute force. Yetch.

If PostgreSQL can pass all three areas of NIST (SQL, ecpg (the pc
directory), and the mc directory) that would be pretty impressive.

-----Original Message-----
From: Jan Wieck [mailto:JanWieck@Yahoo.com]
Sent: Sunday, June 22, 2003 5:45 AM
To: Dann Corbit
Cc: Tom Lane; Jason Earl; PostgreSQL-development
Subject: Re: [HACKERS] Two weeks to feature freeze

Dann Corbit wrote:

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]

Are you volunteering to create it? Step right up.

No. And as an outsider, I rather doubt if any procedures I

developed

would be taken very seriously. If such procedures are to be
developed, I suspect that they will have to be developed

from within

if they are to be successful.

What do you think is the way to become an insider?

Join the CVS tree and make a large and valuable contribution to the
project.

Bruce Momjian writes:

Well, it is a nice test template for people who aren't shell script
experts, and I have been in the habit of pushing stuff I use into /tools
so it is available for others.

I know and I'm not happy about it. The PostgreSQL source tree isn't a
repository of everyone's favorite shell scripts. There's an official
method to build and test a PostgreSQL installation. If that is flawed or
incomplete, then let's talk about it. But everyone pushing out their own
little test methodology without further consideration is not going to help
this discussion.

--
Peter Eisentraut peter_e@gmx.net

"Dann Corbit" <DCorbit@connx.com> writes:

What do you think is the way to become an insider?

Join the CVS tree and make a large and valuable contribution to the
project.

For instance, developing an industrial-strength test suite? If you've
got an itch there, scratch it.

regards, tom lane