It's June 1; do you know where your release is?

Tom,

Let me start this out by voting the things I think we can drop until 8.5:

* gettext plurals patch needs some polishing
-- revert patch, save for 8.5

# adjust information_schema precision and scale fields?
-- save for 8.5

# instrument the Windows shared memory reattachment problem?
-- as much as I'd like to do this, the solution could be as bad as the
problem; we'd need more testing time for new instrumentation. Will have
to deal with via testing patched versions.

# tweak the ordering heuristics for parallel pg_restore?
-- beta2 version is "good enough"; further improvements should be saved
for 8.5.

# change or at least redocument from_collapse_limit?
-- should be doc patch only. Robert Haas should write it.

# revisit increase of default_statistics_target?
-- No. Still appears to be artifact of DBT2. Can't reproduce the issue
using pgbench, or any other test. Still investigating.

Other questions:

# cost_nestloop and cost_hashjoin are broken for SEMI and ANTI joins

* tgl says: I think this is mostly dealt with but it could use
performance testing.

Mark? Jignesh? Can you test this?

--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

On Mon, 2009-06-01 at 11:09 -0700, Josh Berkus wrote:

Tom,

# adjust information_schema precision and scale fields?
-- save for 8.5

I read this thread. It seems for the changes we can make we should just
make them. The actual amount of change is actually very nominal.

Joshua D. Drake

--
PostgreSQL - XMPP: jdrake@jabber.postgresql.org
Consulting, Development, Support, Training
503-667-4564 - http://www.commandprompt.com/
The PostgreSQL Company, serving since 1997

Josh Berkus <josh@agliodbs.com> writes:

# cost_nestloop and cost_hashjoin are broken for SEMI and ANTI joins
* tgl says: I think this is mostly dealt with but it could use performance testing.

Mark? Jignesh? Can you test this?

Actually I'm hoping that Kevin Grittner will have something to report
on that one soon --- he's our best test case for complicated EXISTS
queries. I doubt that the standard benchmarks exercise this code
meaningfully at all.

regards, tom lane

On Mon, Jun 1, 2009 at 2:09 PM, Josh Berkus <josh@agliodbs.com> wrote:

Tom,

Let me start this out by voting the things I think we can drop until 8.5:

* gettext plurals patch needs some polishing
-- revert patch, save for 8.5

#  adjust information_schema precision and scale fields?
-- save for 8.5

# instrument the Windows shared memory reattachment problem?
-- as much as I'd like to do this, the solution could be as bad as the
problem; we'd need more testing time for new instrumentation.  Will have to
deal with via testing patched versions.

# tweak the ordering heuristics for parallel pg_restore?
-- beta2 version is "good enough"; further improvements should be saved for
8.5.

# change or at least redocument from_collapse_limit?
-- should be doc patch only.  Robert Haas should write it.

# revisit increase of default_statistics_target?
-- No.  Still appears to be artifact of DBT2.  Can't reproduce the issue
using pgbench, or any other test.  Still investigating.

Other questions:

#  cost_nestloop and cost_hashjoin are broken for SEMI and ANTI joins

   * tgl says: I think this is mostly dealt with but it could use
performance testing.

Mark?  Jignesh?   Can you test this?

+1 to all of these. Will send extremely short doc patch tonight. I
recommend we create
http://wiki.postgresql.org/wiki/PostgreSQL_8.5_Open_Items and start
bumping things that can't be completed for 8.4.

Basically, my opinion is that if it's not a regression, it's too late
to work on it now. We should ship the release when we're confident
that the new features have had adequate testing and we've squashed all
the regressions.

...Robert

Josh Berkus <josh@agliodbs.com> writes:

Let me start this out by voting the things I think we can drop until 8.5:

* gettext plurals patch needs some polishing
-- revert patch, save for 8.5

Peter might think differently about that ;-). My problem with it is
that we don't seem to have a polished API for how code uses the feature.
I wouldn't mind so much except that once we release we are going to be
stuck with the API indefinitely --- the usage will propagate into
third-party code very quickly and we won't want to break that code by
changing it.

Personally I'd prefer to fix it rather than revert it.

# adjust information_schema precision and scale fields?
-- save for 8.5

No objection here. I mainly wanted to make sure the issue doesn't get
forgotten.

# instrument the Windows shared memory reattachment problem?
-- as much as I'd like to do this, the solution could be as bad as the
problem; we'd need more testing time for new instrumentation. Will have
to deal with via testing patched versions.

Yeah. That was put on the list a month ago, when there was still plenty
of time to do something about it; but since we missed getting it into
beta2 I think it will have to wait.

# tweak the ordering heuristics for parallel pg_restore?
-- beta2 version is "good enough"; further improvements should be saved
for 8.5.

OK, particularly if no one is in a position to test it right away.

regards, tom lane

Robert Haas <robertmhaas@gmail.com> writes:

I recommend we create
http://wiki.postgresql.org/wiki/PostgreSQL_8.5_Open_Items and start
bumping things that can't be completed for 8.4.

Why not just the regular TODO list?

Stuff that has plausible patches attached can go directly to the
CommitFest_2009-First page, but otherwise I don't see a need for
special treatment. If we kick something off the 8.4 open items
list we're essentially saying it doesn't deserve special treatment.

regards, tom lane

On Mon, Jun 1, 2009 at 3:51 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

I recommend we create
http://wiki.postgresql.org/wiki/PostgreSQL_8.5_Open_Items and start
bumping things that can't be completed for 8.4.

Why not just the regular TODO list?

Stuff that has plausible patches attached can go directly to the
CommitFest_2009-First page, but otherwise I don't see a need for
special treatment.  If we kick something off the 8.4 open items
list we're essentially saying it doesn't deserve special treatment.

That would be fine too...

...Robert

Tom Lane wrote:

Josh Berkus <josh@agliodbs.com> writes:

# instrument the Windows shared memory reattachment problem?
-- as much as I'd like to do this, the solution could be as bad as the
problem; we'd need more testing time for new instrumentation. Will have
to deal with via testing patched versions.

Yeah. That was put on the list a month ago, when there was still plenty
of time to do something about it; but since we missed getting it into
beta2 I think it will have to wait.

Agreed. I did some mucking around with it, but the parts I found
reasonably easy to do were also reasonably useless :(

//Magnus

Magnus Hagander <magnus@hagander.net> writes:

Tom Lane wrote:

# instrument the Windows shared memory reattachment problem?

Yeah. That was put on the list a month ago, when there was still plenty
of time to do something about it; but since we missed getting it into
beta2 I think it will have to wait.

Agreed. I did some mucking around with it, but the parts I found
reasonably easy to do were also reasonably useless :(

OK, pushed that item to TODO.

regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> wrote:

Josh Berkus <josh@agliodbs.com> writes:

cost_nestloop and cost_hashjoin are broken for SEMI and ANTI joins

Actually I'm hoping that Kevin Grittner will have something to
report on that one soon

So far, I haven't found any performance regressions in the beta2
release compared to the snapshot from March 2nd on our challenging
queries. They both perform as well or better than 8.3 for our
queries, although there is a slight increase in planning time to get
to the better plans.

Since there are new plans for most of these, I have had to actually
run them to confirm that there is no regression, and that takes some
time. I'm still chipping away, but I would say that it looks good to
me; unless someone else has a query that's behaving badly, I wouldn't
hold it up for this.

I'll post right away if a subsequent test shows a problem.

-Kevin

"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:

Tom Lane <tgl@sss.pgh.pa.us> wrote:

Josh Berkus <josh@agliodbs.com> writes:

cost_nestloop and cost_hashjoin are broken for SEMI and ANTI joins

Actually I'm hoping that Kevin Grittner will have something to
report on that one soon

So far, I haven't found any performance regressions in the beta2
release compared to the snapshot from March 2nd on our challenging
queries. They both perform as well or better than 8.3 for our
queries, although there is a slight increase in planning time to get
to the better plans.

Thanks; I'll remove this from the Open Items list for the moment.

regards, tom lane

"Joshua D. Drake" <jd@commandprompt.com> writes:

On Mon, 2009-06-01 at 11:09 -0700, Josh Berkus wrote:

# adjust information_schema precision and scale fields?
-- save for 8.5

I read this thread. It seems for the changes we can make we should just
make them. The actual amount of change is actually very nominal.

I think the major argument against "just change it" is that we do not
wish to force an initdb now for beta testers, but if we don't there
is always going to be this niggling doubt about how an alleged "8.4"
installation will actually behave. Although I previously suggested
we could live with that, on reflection I don't think that the problem
is important enough to justify it. The information_schema has had
this issue since day one, and we hadn't gotten complaints before.
So pushing it to 8.5 seems the best decision to me.

regards, tom lane

On Mon, Jun 1, 2009 at 12:32 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

As of today we are three months behind the original plan for 8.4.0 release.
In a one-year release cycle that's already pretty bad slip; but there now
seems no chance of a release happening in less than a month, and if we
continue to let things drift it could easily stretch to five or six
months' slip.  Given the slow pace of bug reports there is no reason to
be waiting.  We need to refocus our energy on getting the release out.

The main thing that needs to happen now is to deal with the open items
listed at
http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
either by fixing them or by agreeing that it's okay to let them slide
to 8.5 or beyond.

Regarding this item:

* Consider reverting preventing regular users from base type creation

You raise this point:

tgl says: whether or not we think PL/Java is bulletproof, there are
other problems, for instance this one
http://archives.postgresql.org/message-id/87zlnwnvjg.fsf@news-spur.riddles.org.uk

That's a pretty overwhelming argument for leaving it as-is. I think
we should remove this from the list of open items.

...Robert

On Mon, 1 Jun 2009, Robert Haas wrote:

Regarding this item:

* Consider reverting preventing regular users from base type creation

You raise this point:

tgl says: whether or not we think PL/Java is bulletproof, there are
other problems, for instance this one
http://archives.postgresql.org/message-id/87zlnwnvjg.fsf@news-spur.riddles.org.uk

That's a pretty overwhelming argument for leaving it as-is. I think
we should remove this from the list of open items.

Yes, that makes sense to me as the original requester of this open item.
I thought it had been taken off a while ago.

Kris Jurka

Kris Jurka <books@ejurka.com> writes:

On Mon, 1 Jun 2009, Robert Haas wrote:

tgl says: whether or not we think PL/Java is bulletproof, there are
other problems, for instance this one
http://archives.postgresql.org/message-id/87zlnwnvjg.fsf@news-spur.riddles.org.uk

That's a pretty overwhelming argument for leaving it as-is. I think
we should remove this from the list of open items.

Yes, that makes sense to me as the original requester of this open item.
I thought it had been taken off a while ago.

Removed now.

regards, tom lane

Tom, all,

More suggested dispositions:

* plperl fails with Perl 5.10 on Windows
o tgl says: no reports of this with pre-8.4 Postgres, but I
bet that's just because no one tried it
o dpage says: I'm rolling back the Windows installers to use
5.8 for now. Would appreciate help from anyone familiar with Perl
internals to try to debug this further!

-- Dunstan, Wheeler, Sabino-Mullaine, 'lil help please?

* contrib/seg and contrib/cube GiST index support have performance
issues
o proposed (incomplete) patch here

-- If it's just a performance issue, I don't think this issue should
block 8.4.0; it can be fixed in 8.4.1 if necessary, since we'll probably
want to backpatch the fix anyway.

* possible bug in cover density ranking?

-- From Teodor's response, this is maybe a doc patch and not a code
patch. Teodor? Oleg?

* localeconv encoding issues
o proposed patch here

-- Any reason not to apply patch?

* BUG #4622: xpath only work in utf-8 server encoding
o tgl says: there's a proposed patch for this, but I don't
think it fixes it

-- I think this is a doc patch. Since libxml (as I understand it) only
supports UTF, this is not something we can fix without major engineering
anyway, certainly not before release. I just think we need big warnings
in the docs in several places.

* contrib/intarray opclass definition needs updating
o tgl says: done, but there's another problem; see also bug
#4806

-- This is a serious issue which I'm not sure how we can resolve in the
next couple weeks. Simply throwing a warning is inadequate (although if
we can't fix it in time, we'll have to do that). Having the planner
refuse to use the index if '{}' is involved is problematic from a
performance standpoint. And changing GIN and GiST so they index empty
arrays seems likely to have other side effects. Ideas, anyone?

* Path separator consistency on Windows

-- This discussion does not appear to have concluded. Magnus, Dave?

* autovacuum can run rebuild_database_list unreasonably often

-- A possible quick workaround would be to put a lower limit of naptime
at 1s. This would save most people (those with 10 or less database)
from triggering rebuild_database_list too often. However, given that
it's precisely the people with 100's of databases who would want to
lower naptime to very low levels, this isn't much of a solution.
On the other-other hand, this is enough of a corner case that I think
we can put in a documentation warning and put a fix for this in the TODO
list. Unless Alvaro can get in a patch which prevents
rebuild_database_list from running more often than once per minute this
week?

--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

Josh Berkus wrote:

* plperl fails with Perl 5.10 on Windows
o tgl says: no reports of this with pre-8.4 Postgres, but I
bet that's just because no one tried it
o dpage says: I'm rolling back the Windows installers to use
5.8 for now. Would appreciate help from anyone familiar with Perl
internals to try to debug this further!

-- Dunstan, Wheeler, Sabino-Mullaine, 'lil help please?

I'm working on it.

cheers

andrew

On Tue, Jun 2, 2009 at 8:26 PM, Josh Berkus <josh@agliodbs.com> wrote:

   * autovacuum can run rebuild_database_list unreasonably often

-- A possible quick workaround would be to put a lower limit of naptime at
1s.  This would save most people (those with 10 or less database) from
triggering rebuild_database_list too often.  However, given that it's
precisely the people with 100's of databases who would want to lower naptime
to very low levels, this isn't much of a solution.
 On the other-other hand, this is enough of a corner case that I think we
can put in a documentation warning and put a fix for this in the TODO list.
 Unless Alvaro can get in a patch which prevents rebuild_database_list from
running more often than once per minute this week?

Assuming I have parsed this correctly, +1. I don't think this
particular issue deserves special treatment compared to anything else.

...Robert

On Jun 2, 2009, at 5:26 PM, Josh Berkus wrote:

Tom, all,

More suggested dispositions:

* plperl fails with Perl 5.10 on Windows
o tgl says: no reports of this with pre-8.4 Postgres, but I
bet that's just because no one tried it
o dpage says: I'm rolling back the Windows installers to
use 5.8 for now. Would appreciate help from anyone familiar with
Perl internals to try to debug this further!

-- Dunstan, Wheeler, Sabino-Mullaine, 'lil help please?

I don't think that any of use uses Perl on Windows. But do remember
that there are (at least) two Perls for Windows: ActivePerl and
Strawberry. The latter is the community's choice going forward, and I
suspect it'd be easier to use with Pg, since it's more like the
standard Unix Perl than ActivePerl is.

But someone who's familiar with Perl on Windows needs to look into
this; I don't even have a Windows box.

Best,

David