Remaining 9.5 open items

On 2015-11-30 14:43:59 -0500, Tom Lane wrote:

* pg_rewind exiting with error code 1 when source and target are on the same timeline

Is this a new-in-9.5 bug, or a pre-existing problem? If the latter,
I'm not sure it's a release blocker.

pg_rewind was only introduced in 9.5, no?

* Finish multixact truncation rework

We're not seriously going to push something this large into 9.5 at this
point, are we?

To my knowledge this mostly comment changes. And some, actually
independent, improvements. More blocked on procedual disagreements -
with Noah wanting to revert the existing commits, fixup some stuff, then
reapply them - and me just wanting to do the improvements independently.

Greetings,

Andres Freund

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Tom Lane wrote:

* DDL deparsing testing module should have detected that transforms were not supported, but it failed to notice that

Is this really a release blocker? As a testing matter, it seems like any
fix would go into HEAD only.

Not a blocker as far as I'm concerned.

* another strange behavior with track_commit_timestamp

Where are we on this?

I will test the patch I proposed more thoroughly, then push. I intend
to add a few test cases on top of the recovery testing patch Michael
submitted (but that's branch master only), to make sure I don't
re-introduce bugs already fixed.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

* Tom Lane (tgl@sss.pgh.pa.us) wrote:

Well, it's December nearly, and we don't seem to be making much progress
towards pushing out 9.5.0. I see the following items on
https://wiki.postgresql.org/wiki/PostgreSQL_9.5_Open_Items

* Open Row-Level Security Issues

Seems like what's left here is only documentation fixes, but they still
need to get done.

These are mainly just documentation improvements which I'm working on,
though the docs were recently updated and I need to incorporate Peter's
changes which I wasn't exactly anticipating.

The non-documentation question is around DROP OWNED. We need to either
have policies dropped by DROP OWNED (well, roles removed, unless it's
the last one, in which case the policy should be dropped), or update the
documentation to reflect that they don't. I had been thinking we'd
fix DROP OWNED to deal with the policies, but if folks feel it's too
late for that kind of a change, then we can simply document it. I don't
believe that's unreasonable for a new feature and we can work to get it
addressed in 9.6.

I'm starting to think that just documenting it makes sense for 9.5. I
doubt it's going to be a serious issue during 9.5's lifetime.

Thanks!

Stephen

Stephen Frost wrote:

The non-documentation question is around DROP OWNED. We need to either
have policies dropped by DROP OWNED (well, roles removed, unless it's
the last one, in which case the policy should be dropped), or update the
documentation to reflect that they don't. I had been thinking we'd
fix DROP OWNED to deal with the policies, but if folks feel it's too
late for that kind of a change, then we can simply document it. I don't
believe that's unreasonable for a new feature and we can work to get it
addressed in 9.6.

DROP OWNED is documented as a mechanism to help you drop the role, so
it should do whatever is needed for that. I don't think documenting the
fact that it leaves the user as part of policies is good enough.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:

Stephen Frost wrote:

The non-documentation question is around DROP OWNED. We need to either
have policies dropped by DROP OWNED (well, roles removed, unless it's
the last one, in which case the policy should be dropped), or update the
documentation to reflect that they don't. I had been thinking we'd
fix DROP OWNED to deal with the policies, but if folks feel it's too
late for that kind of a change, then we can simply document it. I don't
believe that's unreasonable for a new feature and we can work to get it
addressed in 9.6.

DROP OWNED is documented as a mechanism to help you drop the role, so
it should do whatever is needed for that. I don't think documenting the
fact that it leaves the user as part of policies is good enough.

We already can't take care of everything with DROP OWNED though, since
we can't do cross-database queries, and the overall process almost
certainly requires additional effort (to reassign objects, etc...), so
while I'd be happier if policies were handled by it, I don't think it's
as serious of an issue.

Still, I'll get a patch worked up for it and then we can discuss the
merits of that patch going in to 9.5 now versus just into HEAD.

Thanks!

Stephen

Stephen Frost wrote:

* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:

Stephen Frost wrote:

The non-documentation question is around DROP OWNED. We need to either
have policies dropped by DROP OWNED (well, roles removed, unless it's
the last one, in which case the policy should be dropped), or update the
documentation to reflect that they don't. I had been thinking we'd
fix DROP OWNED to deal with the policies, but if folks feel it's too
late for that kind of a change, then we can simply document it. I don't
believe that's unreasonable for a new feature and we can work to get it
addressed in 9.6.

DROP OWNED is documented as a mechanism to help you drop the role, so
it should do whatever is needed for that. I don't think documenting the
fact that it leaves the user as part of policies is good enough.

We already can't take care of everything with DROP OWNED though, since
we can't do cross-database queries, and the overall process almost
certainly requires additional effort (to reassign objects, etc...), so
while I'd be happier if policies were handled by it, I don't think it's
as serious of an issue.

Yes, the documentation says to apply a combination of REASSIGN OWNED
plus DROP OWNED to each database. Sure, it's not a single command, but
if you additionally put the burden that the policies must be taken care
of separately, then the whole process is made a little worse.

Still, I'll get a patch worked up for it and then we can discuss the
merits of that patch going in to 9.5 now versus just into HEAD.

Cool.

In the past, we've made a bunch of changes to DROP OWNED in order to
deal with object types that caused errors, even in minor releases. I
think this is just another case of the same problem.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Dec 1, 2015 at 4:43 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

* pg_rewind exiting with error code 1 when source and target are on the same timeline

Is this a new-in-9.5 bug, or a pre-existing problem? If the latter,
I'm not sure it's a release blocker.

pg_rewind has been introduced in 9.5. It would be good to get
something consistent before GA.

* Relation files of unlogged relation for btree and spgist indexes not initialized after promotion

Again, is this a release blocker? It's evidently a very old bug.

Actually for 9.5 it could be said so. The approach proposed by Andres
and that I have patched requires a bump of the WAL format. It would be
nice to get that into 9.5 tree. Or 9.5 will have to use a solution
similar to the back branches, which would be something like syncing
unconditionally INIT_FORKNUM at replay after replaying its FPW.
Thoughts welcome on the dedicated thread.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Nov 30, 2015 at 2:43 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

* Foreign join pushdown vs EvalPlanQual

Is this fixed by 5fc4c26db? If not, what remains to do?

Unfortunately, no. That commit allows FDWs to do proper EPQ handling
for plain table scans, but it proves to be inadequate for EPQ handling
for joins. Solving that problem will require another patch, and,
modulo a bunch of cosmetic issues, I'm reasonably happy with KaiGai
Kohei's latest submission. I'll respond in more detail on that
thread, but the question I want to raise here is: do we want to
back-patch those changes to 9.5 at this late date?

If we don't, then join pushdown won't be usable in 9.5 for queries
that have locking clauses; it will crash. However, it may be that
nobody's going to try to do that anyway. And if they do, they have
the PlannerInfo available when generating paths, so they can just not
push down any joins when there are row marks, which doesn't sound like
the worst thing that ever happened to anybody. It's also possible
that the fix isn't really correct and we won't find that out until
after release, at which point it'll be too late to tinker with the API
(if it isn't already). On the other hand, what do we get out of
shipping an API that we know to be a few bricks short of a load? I
think the risk of collateral damage is low. If there's a problem, I
expect it to be that the join-pushdown-vs-EPQ problem is still not
solved, not that anything else stops working.

From the point of view of existing FDWs, the most noticeable effect of
the patch is that you'll have to pass one more NULL argument to
functions make_foreignscan(). This will break compiles, but it should
be a more trivial repair than what 5fc4c26db demanded. So maybe it's
OK.

I could go either way on this. What do others think?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Nov 30, 2015 at 8:43 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

* Finish multixact truncation rework

We're not seriously going to push something this large into 9.5 at this
point, are we?

I don't know all the details here, so my apologies if any of this is
incorrect/stupid/misinformed.

Given all the quite serious data corruption issues related to
multixact, wouldn't it be motivated to wait releasing 9.5.0 until this
much needed multixct truncation rework has been finished?

This is of course not an issue for users already on >=9.3, since the
problems started in 9.3 and those already on 9.3 and 9.4 already face
the risk.

But users who will consider upgrading from previous versions, will
have to ask themselves if all the new features in 9.5 are worth the
extra risk of data corruption due to the multixact issues.

We at Trustly are still running 9.1 and have been waiting for the
multixact problems to be fixed, which is why we didn't upgrade to 9.4,
and now when I read this I feel really sad we probably have to wait
for 9.6, unless we can accept some increase risk of data corruption.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-12-02 12:14:42 +0100, Joel Jacobson wrote:

On Mon, Nov 30, 2015 at 8:43 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

* Finish multixact truncation rework

We're not seriously going to push something this large into 9.5 at this
point, are we?

I don't know all the details here, so my apologies if any of this is
incorrect/stupid/misinformed.

Given all the quite serious data corruption issues related to
multixact, wouldn't it be motivated to wait releasing 9.5.0 until this
much needed multixct truncation rework has been finished?

The significant changes are in 9.5.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Dec 2, 2015 at 12:19 PM, Andres Freund <andres@anarazel.de> wrote:

The significant changes are in 9.5.

Will multixact truncations be WAL logged in 9.5?

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-12-02 12:25:37 +0100, Joel Jacobson wrote:

On Wed, Dec 2, 2015 at 12:19 PM, Andres Freund <andres@anarazel.de> wrote:

The significant changes are in 9.5.

Will multixact truncations be WAL logged in 9.5?

Yes.

C.f. the release notes:

* Rework truncation of the multixact commit log to be properly WAL-logged
(Andres Freund)

This makes things substantially simpler and more robust.

The relevant commits are

commit aa29c1ccd9f785f9365809f5133e5491acc7ae53
Author: Andres Freund <andres@anarazel.de>
Date: 2015-09-26 19:04:25 +0200

Remove legacy multixact truncation support.
...

Backpatch: 9.5

and
commit 4f627f897367f15702d59973f75f6391d5d3e06f
Author: Andres Freund <andres@anarazel.de>
Date: 2015-09-26 19:04:25 +0200

Rework the way multixact truncations work.
...

Backpatch: 9.5

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Dec 2, 2015 at 12:36 PM, Andres Freund <andres@anarazel.de> wrote:

On 2015-12-02 12:25:37 +0100, Joel Jacobson wrote:

On Wed, Dec 2, 2015 at 12:19 PM, Andres Freund <andres@anarazel.de> wrote:

The significant changes are in 9.5.

Will multixact truncations be WAL logged in 9.5?

Yes.

C.f. the release notes

Excellent! :-) Many many thanks for your efforts! By far the most
important change in 9.5. So much looking forward to upgrade.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Nov 30, 2015 at 4:55 PM, Stephen Frost <sfrost@snowman.net> wrote:

* Tom Lane (tgl@sss.pgh.pa.us) wrote:

Well, it's December nearly, and we don't seem to be making much progress
towards pushing out 9.5.0. I see the following items on
https://wiki.postgresql.org/wiki/PostgreSQL_9.5_Open_Items

* Open Row-Level Security Issues

Seems like what's left here is only documentation fixes, but they still
need to get done.

These are mainly just documentation improvements which I'm working on,
though the docs were recently updated and I need to incorporate Peter's
changes which I wasn't exactly anticipating.

So, when do you foresee this documentation stuff getting taken care
of? We kinda need to do a release here. Is this really a blocker?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/02/2015 05:27 AM, Robert Haas wrote:

On Mon, Nov 30, 2015 at 4:55 PM, Stephen Frost <sfrost@snowman.net> wrote:

* Tom Lane (tgl@sss.pgh.pa.us) wrote:

Well, it's December nearly, and we don't seem to be making much progress
towards pushing out 9.5.0. I see the following items on
https://wiki.postgresql.org/wiki/PostgreSQL_9.5_Open_Items

* Open Row-Level Security Issues

Seems like what's left here is only documentation fixes, but they still
need to get done.

These are mainly just documentation improvements which I'm working on,
though the docs were recently updated and I need to incorporate Peter's
changes which I wasn't exactly anticipating.

So, when do you foresee this documentation stuff getting taken care
of? We kinda need to do a release here. Is this really a blocker?

A feature does not exist without documentation.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-12-02 08:25:13 -0800, Joshua D. Drake wrote:

A feature does not exist without documentation.

Uh, you do realize there's actually documentation about RLS? The issues
mentioned here are some small adjustments, not entirely new docs.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/02/2015 08:39 AM, Andres Freund wrote:

On 2015-12-02 08:25:13 -0800, Joshua D. Drake wrote:

A feature does not exist without documentation.

Uh, you do realize there's actually documentation about RLS? The issues
mentioned here are some small adjustments, not entirely new docs.

No I didn't. I apologize for the noise.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Dec 01, 2015 at 11:05:47AM -0500, Robert Haas wrote:

On Mon, Nov 30, 2015 at 2:43 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

* Foreign join pushdown vs EvalPlanQual

Is this fixed by 5fc4c26db? If not, what remains to do?

Unfortunately, no. That commit allows FDWs to do proper EPQ handling
for plain table scans, but it proves to be inadequate for EPQ handling
for joins. Solving that problem will require another patch, and,
modulo a bunch of cosmetic issues, I'm reasonably happy with KaiGai
Kohei's latest submission. I'll respond in more detail on that
thread, but the question I want to raise here is: do we want to
back-patch those changes to 9.5 at this late date?

Yes. If 9.5 added a bad interface, better to fix the interface even now than
to live with the bad one.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Dec 2, 2015 at 5:27 AM, Robert Haas <robertmhaas@gmail.com> wrote:

These are mainly just documentation improvements which I'm working on,
though the docs were recently updated and I need to incorporate Peter's
changes which I wasn't exactly anticipating.

So, when do you foresee this documentation stuff getting taken care
of? We kinda need to do a release here. Is this really a blocker?

If it's a blocker, I can write the documentation myself. Let's just
fix it, rather than discussing whether or not it needs to be a blocker
-- it will take less time.

Stephen?

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers