Remove vacuum_defer_cleanup_age

On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus <josh@agliodbs.com> wrote:

Given that hot_standby_feedback is pretty bulletproof now, and a lot of
the work in reducing replay conflicts, I think the utility of
vacuum_defer_cleanup_age is at an end. I really meant so submit a patch
to remove it to 9.6, but it got away from me.

Any objections to removing the option in 10?

I'm not sure I see the point.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/12/2016 05:00 PM, Robert Haas wrote:

On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus <josh@agliodbs.com> wrote:

Given that hot_standby_feedback is pretty bulletproof now, and a lot of
the work in reducing replay conflicts, I think the utility of
vacuum_defer_cleanup_age is at an end. I really meant so submit a patch
to remove it to 9.6, but it got away from me.

Any objections to removing the option in 10?

I'm not sure I see the point.

Redusing the number of configuration variables is an a-priori good. In
aggregate, the more knobs we have, the harder it is to learn how to
admin Postgres. Therefore any time a config variable becomes obsolete,
we should remove it.

--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Oct 18, 2016 at 4:18 PM, Josh Berkus <josh@agliodbs.com> wrote:

On 10/12/2016 05:00 PM, Robert Haas wrote:

On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus <josh@agliodbs.com> wrote:

Given that hot_standby_feedback is pretty bulletproof now, and a lot of
the work in reducing replay conflicts, I think the utility of
vacuum_defer_cleanup_age is at an end. I really meant so submit a patch
to remove it to 9.6, but it got away from me.

Any objections to removing the option in 10?

I'm not sure I see the point.

Redusing the number of configuration variables is an a-priori good. In
aggregate, the more knobs we have, the harder it is to learn how to
admin Postgres. Therefore any time a config variable becomes obsolete,
we should remove it.

Meh. I agree that more configuration knobs makes it harder to learn
to configure the system, but we've got enough of them that removing
exactly one isn't going to make a material difference. Against that,
if you are wrong about it being obsolete and there are actually people
relying on it heavily, those people will be very sad if we remove it,
and unless they read this mailing list, we probably won't find out
until it's too late.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/18/2016 01:28 PM, Robert Haas wrote:

On Tue, Oct 18, 2016 at 4:18 PM, Josh Berkus <josh@agliodbs.com> wrote:

On 10/12/2016 05:00 PM, Robert Haas wrote:

On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus <josh@agliodbs.com> wrote:

Given that hot_standby_feedback is pretty bulletproof now, and a lot of
the work in reducing replay conflicts, I think the utility of
vacuum_defer_cleanup_age is at an end. I really meant so submit a patch
to remove it to 9.6, but it got away from me.

Any objections to removing the option in 10?

I'm not sure I see the point.

Redusing the number of configuration variables is an a-priori good. In
aggregate, the more knobs we have, the harder it is to learn how to
admin Postgres. Therefore any time a config variable becomes obsolete,
we should remove it.

Meh. I agree that more configuration knobs makes it harder to learn
to configure the system, but we've got enough of them that removing
exactly one isn't going to make a material difference. Against that,
if you are wrong about it being obsolete and there are actually people
relying on it heavily, those people will be very sad if we remove it,
and unless they read this mailing list, we probably won't find out
until it's too late.

Based on that argument, we would never be able to remove any
configuration parameter ever.

--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Hi,

On 2016-10-09 21:51:07 -0700, Josh Berkus wrote:

Given that hot_standby_feedback is pretty bulletproof now, and a lot of
the work in reducing replay conflicts, I think the utility of
vacuum_defer_cleanup_age is at an end. I really meant so submit a patch
to remove it to 9.6, but it got away from me.

HS feedback doesn't e.g. work well with delayed and/or archived replay,
whereas defer_cleanup does.

On the other hand, removing it would make some of the reasoning around
GetOldestXmin() a bit easier.

Andres

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/18/2016 01:37 PM, Andres Freund wrote:

Hi,

On 2016-10-09 21:51:07 -0700, Josh Berkus wrote:

Given that hot_standby_feedback is pretty bulletproof now, and a lot of
the work in reducing replay conflicts, I think the utility of
vacuum_defer_cleanup_age is at an end. I really meant so submit a patch
to remove it to 9.6, but it got away from me.

HS feedback doesn't e.g. work well with delayed and/or archived replay,
whereas defer_cleanup does.

Oh, point! See, that's why I polled, I knew there was something I was
forgetting about.

On the other hand, removing it would make some of the reasoning around
GetOldestXmin() a bit easier.

Enough to make it worth breaking the above?

--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkus <josh@agliodbs.com> wrote:

Based on that argument, we would never be able to remove any
configuration parameter ever.

Well... no. Based on that argument, we should only remove
configuration parameters if we're fairly certain that they are not
useful any more, which will be rare, but is not never. I agree that
*if* vacuum_defer_cleanup_age is no longer useful, it should be
removed. I'm just not convinced that it's truly obsolete, and you
haven't really offered much of an argument for that proposition. It
does something sufficiently different from hot_standby_feedback that
I'm not sure it's accurate to say that one can substitute for the
other, and indeed, I see Andres has already suggested some scenarios
where it could still be useful.

Actually, I think vacuum_defer_cleanup_age is, and always has been, an
ugly hack. But for some people it may be the ugly hack that is
letting them continue to use PostgreSQL.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote:

On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkus <josh@agliodbs.com> wrote:

Based on that argument, we would never be able to remove any
configuration parameter ever.

Well... no. Based on that argument, we should only remove
configuration parameters if we're fairly certain that they are not
useful any more, which will be rare, but is not never. I agree that
*if* vacuum_defer_cleanup_age is no longer useful, it should be
removed. I'm just not convinced that it's truly obsolete, and you
haven't really offered much of an argument for that proposition. It
does something sufficiently different from hot_standby_feedback that
I'm not sure it's accurate to say that one can substitute for the
other, and indeed, I see Andres has already suggested some scenarios
where it could still be useful.

Actually, I think vacuum_defer_cleanup_age is, and always has been, an
ugly hack. But for some people it may be the ugly hack that is
letting them continue to use PostgreSQL.

I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby
servers --- it cancels transactions rather than delaying cleanup.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Oct 19, 2016 at 8:47 AM, Bruce Momjian <bruce@momjian.us> wrote:

On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote:

On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkus <josh@agliodbs.com> wrote:

Based on that argument, we would never be able to remove any
configuration parameter ever.

Well... no. Based on that argument, we should only remove
configuration parameters if we're fairly certain that they are not
useful any more, which will be rare, but is not never. I agree that
*if* vacuum_defer_cleanup_age is no longer useful, it should be
removed. I'm just not convinced that it's truly obsolete, and you
haven't really offered much of an argument for that proposition. It
does something sufficiently different from hot_standby_feedback that
I'm not sure it's accurate to say that one can substitute for the
other, and indeed, I see Andres has already suggested some scenarios
where it could still be useful.

Actually, I think vacuum_defer_cleanup_age is, and always has been, an
ugly hack. But for some people it may be the ugly hack that is
letting them continue to use PostgreSQL.

I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby
servers --- it cancels transactions rather than delaying cleanup.

I think it's the opposite, isn't it? vacuum_defer_cleanup_age
prevents cancellations.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Oct 19, 2016 at 09:00:06AM -0400, Robert Haas wrote:

On Wed, Oct 19, 2016 at 8:47 AM, Bruce Momjian <bruce@momjian.us> wrote:

On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote:

On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkus <josh@agliodbs.com> wrote:

Based on that argument, we would never be able to remove any
configuration parameter ever.

Well... no. Based on that argument, we should only remove
configuration parameters if we're fairly certain that they are not
useful any more, which will be rare, but is not never. I agree that
*if* vacuum_defer_cleanup_age is no longer useful, it should be
removed. I'm just not convinced that it's truly obsolete, and you
haven't really offered much of an argument for that proposition. It
does something sufficiently different from hot_standby_feedback that
I'm not sure it's accurate to say that one can substitute for the
other, and indeed, I see Andres has already suggested some scenarios
where it could still be useful.

Actually, I think vacuum_defer_cleanup_age is, and always has been, an
ugly hack. But for some people it may be the ugly hack that is
letting them continue to use PostgreSQL.

I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby
servers --- it cancels transactions rather than delaying cleanup.

I think it's the opposite, isn't it? vacuum_defer_cleanup_age
prevents cancellations.

Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms
of xids, that a standby query can run before cancel, like
old_snapshot_threshold, no? After that, we can cancel standby queries.
I see hot_standby_feedback as our current behavior on the master where
we never cancel standby queries.

To me, hot_standby_feedback extends no-cleanup-no-cancel from the
standby to the master, while vacuum_defer_cleanup_age behaves like
old_snapshot_threshold in that it causes cancel for long-running
queries.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 10/19/2016 09:59 AM, Bruce Momjian wrote:

On Wed, Oct 19, 2016 at 09:00:06AM -0400, Robert Haas wrote:

On Wed, Oct 19, 2016 at 8:47 AM, Bruce Momjian <bruce@momjian.us> wrote:

On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote:

Actually, I think vacuum_defer_cleanup_age is, and always has been, an
ugly hack. But for some people it may be the ugly hack that is
letting them continue to use PostgreSQL.

I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby
servers --- it cancels transactions rather than delaying cleanup.

I think it's the opposite, isn't it? vacuum_defer_cleanup_age
prevents cancellations.

Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms
of xids, that a standby query can run before cancel, like
old_snapshot_threshold, no? After that, we can cancel standby queries.
I see hot_standby_feedback as our current behavior on the master where
we never cancel standby queries.

To me, hot_standby_feedback extends no-cleanup-no-cancel from the
standby to the master, while vacuum_defer_cleanup_age behaves like
old_snapshot_threshold in that it causes cancel for long-running
queries.

See Andres' response on this thread. He's already covered why the
setting is still useful, but why we might want to remove it anyway.

--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Oct 19, 2016 at 12:59 PM, Bruce Momjian <bruce@momjian.us> wrote:

Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms
of xids, that a standby query can run before cancel, like
old_snapshot_threshold, no?

No, not really. It affects the behavior of the master, not the standby.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Oct 19, 2016 at 08:17:46PM -0400, Robert Haas wrote:

On Wed, Oct 19, 2016 at 12:59 PM, Bruce Momjian <bruce@momjian.us> wrote:

Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms
of xids, that a standby query can run before cancel, like
old_snapshot_threshold, no?

No, not really. It affects the behavior of the master, not the standby.

But it controls when/if cancels happen on the standby.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Wed, Oct 19, 2016 at 9:09 PM, Bruce Momjian <bruce@momjian.us> wrote:

On Wed, Oct 19, 2016 at 08:17:46PM -0400, Robert Haas wrote:

On Wed, Oct 19, 2016 at 12:59 PM, Bruce Momjian <bruce@momjian.us> wrote:

Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms
of xids, that a standby query can run before cancel, like
old_snapshot_threshold, no?

No, not really. It affects the behavior of the master, not the standby.

But it controls when/if cancels happen on the standby.

True. I see your point.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers