9.3 Beta1 status report

On Sat, Apr 20, 2013 at 10:02 PM, Bruce Momjian <bruce@momjian.us> wrote:

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup.

I've noticed a few things:

* Allow heap-only tuple updates on system tables (Andres Freund)

Didn't Andres just fix a bug wherein HOT updates usually wouldn't
occur on system tables following the commit of the foreign key locks
patch? While HOT's development occurred at a time before I followed
pgsql-hackers, I seem to recall someone telling me that Tom insisted
upon HOT working with system catalogs specifically because if it
wasn't good enough to work there, it wasn't good enough to work
anywhere, or something like that. I guess the source of the confusion
is specifically that at one point HOT really didn't work with system
catalogs. But, if I'm not mistaken, never in a released version.

* Improve grouping of sessions waiting for commit_delay (Peter Geoghegan)

I think this should be under "General Performance". It's definitely a
performance feature.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Sun, Apr 21, 2013 at 11:06 AM, Peter Geoghegan <pg@heroku.com> wrote:

On Sat, Apr 20, 2013 at 10:02 PM, Bruce Momjian <bruce@momjian.us> wrote:

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup.

I've noticed a few things:

* Allow heap-only tuple updates on system tables (Andres Freund)

Didn't Andres just fix a bug wherein HOT updates usually wouldn't
occur on system tables following the commit of the foreign key locks
patch? While HOT's development occurred at a time before I followed
pgsql-hackers, I seem to recall someone telling me that Tom insisted
upon HOT working with system catalogs specifically because if it
wasn't good enough to work there, it wasn't good enough to work
anywhere, or something like that. I guess the source of the confusion
is specifically that at one point HOT really didn't work with system
catalogs. But, if I'm not mistaken, never in a released version.

Yeah, HOT was always supported on the system tables in the released
versions. Early days, I tried to convince Tom that its OK to not support
HOT on system tables because updating it frequently is not that common. But
he rejected that on the grounds you explained above. Similarly, we had
other limitations such as CREATE INDEX CONCURRENTLY was broken in the early
submitted versions, but Tom insisted on getting that to work as well before
he will consider the patch. In retrospect, even though we had to burn
midnight oil to get all those limitations straight up, IMHO it made the
code more solid. Of course, Tom had a magic eye to find many corner cases,
fix them and simplify the code before committing the patch.

Andreas reported and fixed a bug which was an oversight in the FK locks
patch. I think he also discovered an old bug that the tableoid was not
being properly checked for HOT conditions but that had very limited impact
since its not common to have indexes on tableoids.

Thanks,
Pavan

--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee

2013-04-21 07:02 keltezéssel, Bruce Momjian írta:

I am not sure if Tom shared yet, but we are planning to package 9.3
beta1 on April 29, with a release on May 2. Those dates might change,
but that is the current plan. I have completed a draft 9.3 release
notes, which you can view here:

http://momjian.us/pgsql_docs/release-9-3.html

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup.

How comes Álvaro's name comes out right in your page but not at
http://www.postgresql.org/docs/devel/static/release-9-3.html ?

Anyway, I attached a patch to fix my name in your page using markups.

Thanks,
Zoltán Böszörményi

--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/

On Sun, Apr 21, 2013 at 9:02 AM, Bruce Momjian <bruce@momjian.us> wrote:

I am not sure if Tom shared yet, but we are planning to package 9.3
beta1 on April 29, with a release on May 2. Those dates might change,
but that is the current plan. I have completed a draft 9.3 release
notes, which you can view here:

http://momjian.us/pgsql_docs/release-9-3.html

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup

* Collect and use histograms of lower and upper bounds for range types
(Alexander Korotkov)

Actually, we also collect histogram of range lengths. Probably, we can
rephrase it more generally, like "Collect and use special statistics for
range types".

------
With best regards,
Alexander Korotkov.

On 2013-04-20 22:36:32 -0700, Peter Geoghegan wrote:

On Sat, Apr 20, 2013 at 10:02 PM, Bruce Momjian <bruce@momjian.us> wrote:

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup.

I've noticed a few things:

* Allow heap-only tuple updates on system tables (Andres Freund)

Didn't Andres just fix a bug wherein HOT updates usually wouldn't
occur on system tables following the commit of the foreign key locks
patch?

Yes, that definitely was a bugfix for a HEAD only feature.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Sat, Apr 20, 2013 at 10:36:32PM -0700, Peter Geoghegan wrote:

* Improve grouping of sessions waiting for commit_delay (Peter Geoghegan)

I think this should be under "General Performance". It's definitely a
performance feature.

OK, moved.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Sun, Apr 21, 2013 at 09:34:10AM +0200, Boszormenyi Zoltan wrote:

2013-04-21 07:02 keltez�ssel, Bruce Momjian �rta:

I am not sure if Tom shared yet, but we are planning to package 9.3
beta1 on April 29, with a release on May 2. Those dates might change,
but that is the current plan. I have completed a draft 9.3 release
notes, which you can view here:

http://momjian.us/pgsql_docs/release-9-3.html

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup.

How comes �lvaro's name comes out right in your page but not at
http://www.postgresql.org/docs/devel/static/release-9-3.html ?

Anyway, I attached a patch to fix my name in your page using markups.

Thanks, applied. I had not yet gotten to doing the SGML markup for
non-ASCII characters.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Sun, Apr 21, 2013 at 02:45:42PM +0400, Alexander Korotkov wrote:

On Sun, Apr 21, 2013 at 9:02 AM, Bruce Momjian <bruce@momjian.us> wrote:

I am not sure if Tom shared yet, but we are planning to package 9.3
beta1 on April 29, with a release on May 2. Those dates might change,
but that is the current plan. I have completed a draft 9.3 release
notes, which you can view here:

http://momjian.us/pgsql_docs/release-9-3.html

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup

* Collect and use histograms of lower and upper bounds for range types
(Alexander Korotkov)

Actually, we also collect histogram of range lengths. Probably, we can rephrase
it more generally, like "Collect and use special statistics for range types".

Done, thanks.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Bruce,

I don't see parallel pg_dump in the release notes. I thought that got
committed?

Anyway, see the pgsql-advocacy list for a longish discussion about what
we should consider the "major" fetures for 9.3.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2013-04-21 14:50:07 -0700, Josh Berkus wrote:

Bruce,

I don't see parallel pg_dump in the release notes. I thought that got
committed?

E.1.3.8.2. pg_dump:
Add pg_dump --jobs to dump in parallel when using directory output
format (Joachim Wieland)

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

2013-04-21 15:10 keltezéssel, Bruce Momjian írta:

On Sun, Apr 21, 2013 at 09:34:10AM +0200, Boszormenyi Zoltan wrote:

2013-04-21 07:02 keltezéssel, Bruce Momjian írta:

I am not sure if Tom shared yet, but we are planning to package 9.3
beta1 on April 29, with a release on May 2. Those dates might change,
but that is the current plan. I have completed a draft 9.3 release
notes, which you can view here:

http://momjian.us/pgsql_docs/release-9-3.html

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup.

How comes Álvaro's name comes out right in your page but not at
http://www.postgresql.org/docs/devel/static/release-9-3.html ?

Anyway, I attached a patch to fix my name in your page using markups.

Thanks, applied.

Thank you.

I had not yet gotten to doing the SGML markup for
non-ASCII characters.

--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Bruce Momjian wrote:

I am not sure if Tom shared yet, but we are planning to package 9.3
beta1 on April 29, with a release on May 2. Those dates might change,
but that is the current plan. I have completed a draft 9.3 release
notes, which you can view here:

http://momjian.us/pgsql_docs/release-9-3.html

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup.

Can you please clarify the policy on attaching people's names to items?

This item
Have DROP OWNED only remove user-matching GRANTs on shared objects, e.g.
databases, tablespaces (Álvaro Herrera)
was a backpatched bugfix; I don't think it should be listed.

This item
Throw an error if expiring tuple is again updated or deleted (Kevin
Grittner) KEEP?

not only needs to be kept, but is also a backward-incompatible change,
so I think it warrants a more verbose explanation.

This item
Improve the ability to detect indexable prefixes in regular
expressions (Tom Lane)
I'm not really sure about it. Isn't it about the new pg_trgm code to
support regex indexes? I think they either belong together, or perhaps
the one in "optimizer" shouldn't be listed.

This item
Implement a generic binary heap and use it for Merge-Append operations
(Abhijit Menon-Sen)

A generic binary heap was implemented; but merge-append was already
using their own binary heap. So this is not a performance optimization.
I think the item should be moved down to the "source code" section.

There's an extra double quote here:
"Allow in-memory sorts to use their full memory allocation (Jeff Janes)

This item:
Allow heap-only tuple updates on system tables (Andres Freund)
was a bug fix; item should be removed.

Shouldn't this one
Add function to report the size of the GIN pending index insertion
list (Fujii Masao)
be in the "additional modules" section?

In this item
Add support to event triggers (Dimitri Fontaine, Tom Lane)
I am not sure why you list Tom. I think Robert should be listed
instead.

In this this
Internally store default foreign key matches (non-FULL, non-PARTIAL) as
"simple" (Tom Lane)
there is something funny going on with & chars around unspecified.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Apr 22, 2013 at 01:54:03PM -0300, Alvaro Herrera wrote:

Bruce Momjian wrote:

I am not sure if Tom shared yet, but we are planning to package 9.3
beta1 on April 29, with a release on May 2. Those dates might change,
but that is the current plan. I have completed a draft 9.3 release
notes, which you can view here:

http://momjian.us/pgsql_docs/release-9-3.html

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup.

Can you please clarify the policy on attaching people's names to items?

Well, I just pull them from the commit message, of it no one is
mentioned in the commit message, I use the committer's name.

This item
Have DROP OWNED only remove user-matching GRANTs on shared objects, e.g.
databases, tablespaces (�lvaro Herrera)
was a backpatched bugfix; I don't think it should be listed.

OK, removed. I now see there was a backpatch mention in the commit
messsage.

This item
Throw an error if expiring tuple is again updated or deleted (Kevin
Grittner) KEEP?

not only needs to be kept, but is also a backward-incompatible change,
so I think it warrants a more verbose explanation.

OK, I don't understand myself, so I will need details. I marked it as
backward-incompatible.

This item
Improve the ability to detect indexable prefixes in regular
expressions (Tom Lane)
I'm not really sure about it. Isn't it about the new pg_trgm code to
support regex indexes? I think they either belong together, or perhaps
the one in "optimizer" shouldn't be listed.

I have no idea. I certainly see it affecting more than pg_trgm; I see
backend regression test additions with the patch,
628cbb50ba80c83917b07a7609ddec12cda172d0.

This item
Implement a generic binary heap and use it for Merge-Append operations
(Abhijit Menon-Sen)

A generic binary heap was implemented; but merge-append was already
using their own binary heap. So this is not a performance optimization.
I think the item should be moved down to the "source code" section.

OK.

There's an extra double quote here:
"Allow in-memory sorts to use their full memory allocation (Jeff Janes)

Fixed.

This item:
Allow heap-only tuple updates on system tables (Andres Freund)
was a bug fix; item should be removed.

OK.

Shouldn't this one
Add function to report the size of the GIN pending index insertion
list (Fujii Masao)
be in the "additional modules" section?

Yes, moved.

In this item
Add support to event triggers (Dimitri Fontaine, Tom Lane)
I am not sure why you list Tom. I think Robert should be listed
instead.

Tom did a massive fix/cleanup of that code. I have added Robert.

In this this
Internally store default foreign key matches (non-FULL, non-PARTIAL) as
"simple" (Tom Lane)
there is something funny going on with & chars around unspecified.

Fixed, and applied.

Thanks!

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Allow tooling like pg_receivexlog to run on computers with different architectures (Heikki Linnakangas)

This probably should be mentioned in the backwards-compatibility
section. Any 3rd party tools that speak the streaming replication
protocol are affected.

E.1.3.2.1. Write-Ahead Log (WAL)

Store WAL in a continuous stream, rather than skipping the last 16MB segment every 4GB (Heikki Linnakangas) BACKWARD COMPATIBLE BREAK

Restructure WAL files to better handle timeline changes during recovery (Heikki Linnakangas)

Restructure WAL files to use a more compact storage format (Heikki Linnakangas)

Can you clarify which commits these came from? The first one is clear
(dfda6eba), and I think the 3rd covers commits 20ba5ca6 and 061e7efb1.
But what is that second entry?

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Apr 22, 2013 at 2:33 PM, Bruce Momjian <bruce@momjian.us> wrote:

In this item
Add support to event triggers (Dimitri Fontaine, Tom Lane)
I am not sure why you list Tom. I think Robert should be listed
instead.

Tom did a massive fix/cleanup of that code. I have added Robert.

I do not think that is true. To what commit IDs are you referring? I
think this item should credit Dimitri, myself, and Alvaro, probably in
that order. The only commit by Tom to event_triggers.c is
cd3413ec3683918c9cb9cfb39ae5b2c32f231e8b, which is hardly a "massive
fix/cleanup".

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Bruce Momjian wrote:

On Mon, Apr 22, 2013 at 01:54:03PM -0300, Alvaro Herrera wrote:

Bruce Momjian wrote:

I am not sure if Tom shared yet, but we are planning to package 9.3
beta1 on April 29, with a release on May 2. Those dates might change,
but that is the current plan. I have completed a draft 9.3 release
notes, which you can view here:

http://momjian.us/pgsql_docs/release-9-3.html

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup.

Can you please clarify the policy on attaching people's names to items?

Well, I just pull them from the commit message, of it no one is
mentioned in the commit message, I use the committer's name.

Hm, I listed code authors in roughly chronological order in 0ac5ad51
(fklocks). I think that patch should list me, Noah, Andres, Alex,
Marti.

This item
Improve the ability to detect indexable prefixes in regular
expressions (Tom Lane)
I'm not really sure about it. Isn't it about the new pg_trgm code to
support regex indexes? I think they either belong together, or perhaps
the one in "optimizer" shouldn't be listed.

I have no idea. I certainly see it affecting more than pg_trgm; I see
backend regression test additions with the patch,
628cbb50ba80c83917b07a7609ddec12cda172d0.

Ah, yeah, it's unrelated to pg_trgm indexing. It's a (backpatched) bug
fix, though.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Apr 22, 2013 at 03:19:36PM -0400, Robert Haas wrote:

On Mon, Apr 22, 2013 at 2:33 PM, Bruce Momjian <bruce@momjian.us> wrote:

In this item
Add support to event triggers (Dimitri Fontaine, Tom Lane)
I am not sure why you list Tom. I think Robert should be listed
instead.

Tom did a massive fix/cleanup of that code. I have added Robert.

I do not think that is true. To what commit IDs are you referring? I
think this item should credit Dimitri, myself, and Alvaro, probably in
that order. The only commit by Tom to event_triggers.c is
cd3413ec3683918c9cb9cfb39ae5b2c32f231e8b, which is hardly a "massive
fix/cleanup".

OK, changed.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Apr 22, 2013 at 04:48:58PM -0300, Alvaro Herrera wrote:

Bruce Momjian wrote:

On Mon, Apr 22, 2013 at 01:54:03PM -0300, Alvaro Herrera wrote:

Bruce Momjian wrote:

I am not sure if Tom shared yet, but we are planning to package 9.3
beta1 on April 29, with a release on May 2. Those dates might change,
but that is the current plan. I have completed a draft 9.3 release
notes, which you can view here:

http://momjian.us/pgsql_docs/release-9-3.html

I will be working on polishing them for the next ten days, so any
feedback, patches, or commits are welcome. I still need to add lots of
SGML markup.

Can you please clarify the policy on attaching people's names to items?

Well, I just pull them from the commit message, of it no one is
mentioned in the commit message, I use the committer's name.

Hm, I listed code authors in roughly chronological order in 0ac5ad51
(fklocks). I think that patch should list me, Noah, Andres, Alex,
Marti.

OK, I have now listed them in the order you specified.

This item
Improve the ability to detect indexable prefixes in regular
expressions (Tom Lane)
I'm not really sure about it. Isn't it about the new pg_trgm code to
support regex indexes? I think they either belong together, or perhaps
the one in "optimizer" shouldn't be listed.

I have no idea. I certainly see it affecting more than pg_trgm; I see
backend regression test additions with the patch,
628cbb50ba80c83917b07a7609ddec12cda172d0.

Ah, yeah, it's unrelated to pg_trgm indexing. It's a (backpatched) bug
fix, though.

OK, removed.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Apr 22, 2013 at 10:11:48PM +0300, Heikki Linnakangas wrote:

Allow tooling like pg_receivexlog to run on computers with different architectures (Heikki Linnakangas)

This probably should be mentioned in the backwards-compatibility
section. Any 3rd party tools that speak the streaming replication
protocol are affected.

E.1.3.2.1. Write-Ahead Log (WAL)

Store WAL in a continuous stream, rather than skipping the last 16MB segment every 4GB (Heikki Linnakangas) BACKWARD COMPATIBLE BREAK

Restructure WAL files to better handle timeline changes during recovery (Heikki Linnakangas)

Restructure WAL files to use a more compact storage format (Heikki Linnakangas)

Can you clarify which commits these came from? The first one is
clear (dfda6eba), and I think the 3rd covers commits 20ba5ca6 and
061e7efb1. But what is that second entry?

Frankly, I found the WAL and timeline commits all over the place and
could hardly make sense of it. I tried to collapse entries into
meaningful items, but I need help. Can you suggest changes?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers