Getting to beta1

On Fri, 2010-03-12 at 22:28 -0500, Bruce Momjian wrote:

Where are we in getting to beta1? I know people are looking to me for
9.0 release notes and I will have them done in about a week, but what
about open issues? I don't see many on the main 9.0 open items page:

http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items#Bugs

The list has been reduced greatly in the past week. What about HS/SR
open items?

I have these things on my list

* Minor page xid bug fix
* btree delete standby-side derivation of xid
* review of StartupXLog issue, on open items list, has an effect on HS

I expect to be finished with those by Wed, perhaps Thurs.

Some additional open items on SR

* trigger_file no longer supports "fast" mode. There is no way to
startup server quickly if it has fallen behind, as is currently possible
with 8.4

* There should be a default for "trigger_file" so that you can failover
a server even if this has not been specified.

To me the open items look like at least 2 weeks work on SR, given some
of them will likely need discussion rather than just action.

--
Simon Riggs www.2ndQuadrant.com

The list has been reduced greatly in the past week. What about HS/SR
open items?

I'd like to see vacuum_defer_cleanup_age added to the "Archive" section
of postgresql.conf, and add it to the docs (I'll write something this
week). I'd like to get rid of the associated hint-bits bogus error
messsage.

--Josh Berkus

Simon Riggs wrote:

I have these things on my list

* Minor page xid bug fix
* btree delete standby-side derivation of xid
* review of StartupXLog issue, on open items list, has an effect on HS

I expect to be finished with those by Wed, perhaps Thurs.

Don't forget the "start from shutdown checkpoint" issue.

* There should be a default for "trigger_file" so that you can failover
a server even if this has not been specified.

I disagree. In many cases, you never want to failover the standby, and
having a default would just get in your way.

To me the open items look like at least 2 weeks work on SR, given some
of them will likely need discussion rather than just action.

Yeah, I've unfortunately had very little to no time at all on SR
recently. I must reserve some full days for that again...

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Sun, 2010-03-14 at 16:53 +0200, Heikki Linnakangas wrote:

Simon Riggs wrote:

I have these things on my list

* Minor page xid bug fix
* btree delete standby-side derivation of xid
* review of StartupXLog issue, on open items list, has an effect on HS

I expect to be finished with those by Wed, perhaps Thurs.

Don't forget the "start from shutdown checkpoint" issue.

How could I? :-)

* There should be a default for "trigger_file" so that you can failover
a server even if this has not been specified.

I disagree. In many cases, you never want to failover the standby, and
having a default would just get in your way.

An explanation in the docs would be good. And also a hint of how to
failover if you decide in an emergency that the absence was a mistake,
in retrospect.

To me the open items look like at least 2 weeks work on SR, given some
of them will likely need discussion rather than just action.

Yeah, I've unfortunately had very little to no time at all on SR
recently. I must reserve some full days for that again...

12 open items says "yes please" to that.

--
Simon Riggs www.2ndQuadrant.com

On 3/14/10 9:02 AM, Simon Riggs wrote:

An explanation in the docs would be good. And also a hint of how to
failover if you decide in an emergency that the absence was a mistake,
in retrospect.

I'm planning on writing a "Guide to HS & SR" for the beta. Originally I
planned to put this in the main docs, but I couldn't figure out how to
fit it in there structurally. Plus, it needs more examples, output
samples, and a tutorial feel.

--Josh Berkus

Devs,

Also, I would like to have a Beta or at least a new alpha release before
April 3 for the test-fest, so that our volunteers aren't testing bugs
which are already patched.

--Josh

On Mar 14, 2010, at 3:38 PM, Josh Berkus wrote:

I'm planning on writing a "Guide to HS & SR" for the beta. Originally I
planned to put this in the main docs, but I couldn't figure out how to
fit it in there structurally. Plus, it needs more examples, output
samples, and a tutorial feel.

Perhaps a tutorial could go under Server Administration? Or perhaps under Tutorial even? It would be section I.4.

http://developer.postgresql.org/pgdocs/postgres/index.html

Frankly, I think more examples and tutorials in the docs would help newbies a *lot*.

Best,

David

On Sat, Mar 13, 2010 at 12:28 PM, Bruce Momjian <bruce@momjian.us> wrote:

Where are we in getting to beta1?  I know people are looking to me for
9.0 release notes and I will have them done in about a week, but what
about open issues?  I don't see many on the main 9.0 open items page:

       http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items#Bugs

The list has been reduced greatly in the past week.  What about HS/SR
open items?

I think that at least the following item should be addressed before beta1
since it's a serious problem.

* Walreceiver is not interruptible on win32
http://archives.postgresql.org/pgsql-hackers/2010-01/msg01672.php

I've already submitted the patch, and am waiting for the review.

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

"David E. Wheeler" <david@kineticode.com> writes:

On Mar 14, 2010, at 3:38 PM, Josh Berkus wrote:

I'm planning on writing a "Guide to HS & SR" for the beta. Originally I
planned to put this in the main docs, but I couldn't figure out how to
fit it in there structurally. Plus, it needs more examples, output
samples, and a tutorial feel.

Perhaps a tutorial could go under Server Administration? Or perhaps
under Tutorial even? It would be section I.4.

+1 for having a Tutorial chapter about setting up archiving, PITR and
replication. While at it, let's read the release notes and list other
points that the tutorial could cover too.

Do we need some more sections in Chapter 3. Advanced Features, dealing
with Exclusion Constraint, the new DO command, how to manage privileges
in a realistic examples (a superuser role shared between 2 DBAs, the
database owner, and the application which is not allowed DDL, for
example)?

A psql chapter/section would maybe fit too, with tricks such as \o then
query the catalog then \i, ON_ERROR_{STOP,ROLLBACK}, -1, -v, PGOPTIONS,
etc, like Peter did in his last blog entry.

Maybe some more admin level tutorial would be great to have too, such as
how to find what's locking, how to monitor table and index usage to
determine which indexes to drop, which to create, how to monitor
<things> (slaves lag, hitratio, transactions, I/U/D activity, you name
it).

A lot of things are described in the manual and provided in munin or
nagios plugins already, but still the Tutorial looks like a good place
to give the recipes, ready-to-go queries etc.

Regards,
--
dim

Dimitri Fontaine wrote:

Maybe some more admin level tutorial would be great to have too, such as
how to find what's locking, how to monitor table and index usage to
determine which indexes to drop, which to create, how to monitor
<things> (slaves lag, hitratio, transactions, I/U/D activity, you name
it).

Wow, that's at least one order of magnitude more ambitious than the
actual scope of work on the docs that should be getting focused on for
beta right now, perhaps two. Regardless, I already have stubs for the
first couple of these sitting on the wiki at
http://wiki.postgresql.org/wiki/Category:Administration (locks,
monitoring). I know I'd rather see work done on those, where we can
continue to improve without doc commits and easily make things available
for all versions, until that content is good. Maybe then we can talk
about merging some of that back into the main docs.

--
Greg Smith 2ndQuadrant US Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com www.2ndQuadrant.us

Greg Smith <greg@2ndquadrant.com> writes:

Dimitri Fontaine wrote:

Maybe some more admin level tutorial would be great to have too, such as
how to find what's locking, how to monitor table and index usage to
determine which indexes to drop, which to create, how to monitor
<things> (slaves lag, hitratio, transactions, I/U/D activity, you name
it).

Wow, that's at least one order of magnitude more ambitious than the actual
scope of work on the docs that should be getting focused on for beta right
now, perhaps two.

Yes, I took the message as an opportunity to talk about how much stuff
we'd like to add in the tutorial, then I'll see about spending time on
it if core agrees with the need. There's no reason I'd want this to
happen pre-beta unless it's about hard to grasp things we want lots of
people to test. So we're in agreement here...

Maybe it's time to start another thread if people want to follow-up on
expanding our tutorial.

Regardless, I already have stubs for the first couple of
these sitting on the wiki at
http://wiki.postgresql.org/wiki/Category:Administration (locks, monitoring).
I know I'd rather see work done on those, where we can continue to improve
without doc commits and easily make things available for all versions, until
that content is good. Maybe then we can talk about merging some of that
back into the main docs.

Some kind of canonical reference on how to use the catalogs and system
view to realise common tasks does not seem out of place in the tutorial
for me.

As far as using the wiki to prepare the content, +1.
--
Dimitri Fontaine
PostgreSQL DBA, Architecte

Dimitri Fontaine <dfontaine@hi-media.com> writes:

A lot of things are described in the manual and provided in munin or
nagios plugins already, but still the Tutorial looks like a good place
to give the recipes, ready-to-go queries etc.

This sounds like a pretty horrid idea. The tutorial is meant to be read
first, so it cannot depend on having already read any of the main
documentation. If we try to fill it with "hints and tricks" then either
it will be completely unintelligible to newbies, or there will be a
staggering amount of material duplicated from the main docs to support
the hints. The latter approach would be no fun to write and even less
fun to maintain in the future.

There might well be a use for a separate hints-and-tricks document.
I don't agree with stuffing it into the existing tutorial though.

regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> writes:

This sounds like a pretty horrid idea. The tutorial is meant to be read
first, so it cannot depend on having already read any of the main
documentation. If we try to fill it with "hints and tricks" then either
it will be completely unintelligible to newbies, or there will be a
staggering amount of material duplicated from the main docs to support
the hints. The latter approach would be no fun to write and even less
fun to maintain in the future.

I'm not sure how much your comment applies to what I picture, so here's
what I had in mind:

There's a system view called pg_stat_activity which is maintained
up-to-date by PostgreSQL, and you can query it to gather information
about running queries at any time. Another such view is pg_locks,
which reports about current lock requests and whether they're granted
or not.

You can use both those system views to get important information on
your running system, and to show currently waiting for a lock query
texts, here's how you join those views:

<insert recipe from the link>
http://wiki.postgresql.org/wiki/Lock_Monitoring

To produce a locking situation, let's open two concurrent connections
to the database, either with psql in two different terminals, or using
pgadmin. Now <instruction for 2 concurrent UPDATE on the same
row>. Use the previous query to see the locked query, commit the first
session to unlock it.

As the concepts of SELECT, view and JOINs are already addressed in the
tutorial, I'd think it could be ok. Now, maybe the tutorial isn't the
right place to be confronted to MVCC, locks, and system monitoring, but
some kind of soft introduction would be good here, methinks.

There might well be a use for a separate hints-and-tricks document.
I don't agree with stuffing it into the existing tutorial though.

Yeah, maybe expanding current chapter 27. Monitoring Database Activity
would be a better idea?

http://developer.postgresql.org/pgdocs/postgres/monitoring.html

In fact, a merge of chapters 27 and 28 Monitoring Disk Usage into a
larger one about Monitoring PostgreSQL could be a better fit?

Regards,
--
dim

On 3/15/10 5:47 AM, Dimitri Fontaine wrote:

Maybe it's time to start another thread if people want to follow-up on
expanding our tutorial.

Yes, and on pgsql-docs rather than on this mailing list.

Or ... J.F.D.I (Just F---- Do It). That is, if someone contributed a
whole buncha new text to the tutorial on pgsql-docs, I can't imagine it
being rejected out of hand.

For my part, I plan to just write the tutorial in whatever tool makes it
easiest to write (likely Lyx, but maybe OOo). Then people can discuss
what portions belong in the docs, or not.

--Josh Berkus

Josh Berkus wrote:

Devs,

Also, I would like to have a Beta or at least a new alpha release before
April 3 for the test-fest, so that our volunteers aren't testing bugs
which are already patched.

We can easily create another alpha by April 3. I think the big question
is whether we can put out beta1 while we still have open HS/SR issues.
My guess is no. My other guess is that we will still have open HS/SR
issues on April 3. So, putting those two guesses together, we will
create a new alpha by April 3 for you. :-|

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

PG East: http://www.enterprisedb.com/community/nav-pg-east-2010.do

On Mon, Mar 15, 2010 at 4:24 PM, Bruce Momjian <bruce@momjian.us> wrote:

We can easily create another alpha by April 3.  I think the big question
is whether we can put out beta1 while we still have open HS/SR issues.
My guess is no.  My other guess is that we will still have open HS/SR
issues on April 3.  So, putting those two guesses together, we will
create a new alpha by April 3 for you.  :-|

I think we need to do a better job defining exactly what we think the
"must fix" HS/SR issues are. Otherwise I can see this process of
trying to get to beta dragging out almost indefinitely.

...Robert

Josh Berkus <josh@agliodbs.com> writes:

Yes, and on pgsql-docs rather than on this mailing list.

Or ... J.F.D.I (Just F---- Do It). That is, if someone contributed a
whole buncha new text to the tutorial on pgsql-docs, I can't imagine it
being rejected out of hand.

That was the bulk of the question, thanks for such a clear answer. I'll
see about giving some time to that, out of the critical path to beta.

For my part, I plan to just write the tutorial in whatever tool makes it
easiest to write (likely Lyx, but maybe OOo). Then people can discuss
what portions belong in the docs, or not.

Should I follow you on that choice or just send a doc patch with poor
SGML markup?
--
dim

All,

A user at SFPUG last night pointed out why we should release a beta,
rather than an alpha, sooner rather than later: because there are no
Windows packages for Alphas.

Currently, our Windows users are *not* testing 9.0. Which means we're
just putting off the day when we hear about Windows-specific bugs.

--
-- Josh Berkus
PostgreSQL Experts Inc.
http://www.pgexperts.com

On Wed, Mar 17, 2010 at 4:54 PM, Josh Berkus <josh@agliodbs.com> wrote:

All,

A user at SFPUG last night pointed out why we should release a beta,
rather than an alpha,  sooner rather than later: because there are no
Windows packages for Alphas.

Yes there: http://www.enterprisedb.com/products/pgdevdownload.do

We've produced them since Alpha 2 iirc.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
PG East Conference: http://www.enterprisedb.com/community/nav-pg-east-2010.do