cutting down the TODO list thread

Started by John Naylorover 5 years ago65 messageshackers

john.naylor@enterprisedb.com

over 5 years ago

As I mentioned in [1]/messages/by-id/CAFBsxsHbqMzDoGB3eAGmpcpB+7uae+LLi_G+o8HMEECM9CbQcQ@mail.gmail.com, I've volunteered to clear out the TODO list of items
that appear to be too difficult, controversial, or otherwise not worth
doing to warrant being listed there. I'll be working a few sections at a
time, and every so often I'll have a list of proposed items for removal. If
I don't hear objections, I'll remove the items after a few days while going
through the next set.

Where there's an email thread, I've skimmed a few messages to get a sense
of the community's thoughts on it. Where easily determined, I've taken age
into account, insofar as something from 2017 is going to get much more
benefit of doubt than something from 2008. I've added after each item a
phrase that sums up the reason I believe it doesn't belong anymore.
Feedback welcome, of course, although I suspect there won't be much.

**Administration

- Have custom variables be transaction-safe
Old and found to be difficult after attempting

- Allow custom variables to appear in pg_settings()
Old and controversial

- Implement the SQL-standard mechanism whereby REVOKE ROLE revokes only the
privilege granted by the invoking role, and not those granted by other roles
Old and difficult

- Prevent query cancel packets from being replayed by an attacker,
especially when using SSL
Old and difficult

*Configuration files

- Consider normalizing fractions in postgresql.conf, perhaps using '%'
At the time (2007), some gucs used an actual percentage.

- Add external tool to auto-tune some postgresql.conf parameters
There are already out-of-core tools that try to do this.

- Create utility to compute accurate random_page_cost value
Seems outdated: In the current age of SSDs and cloud environments, it's
often just set to 1.1, and there hasn't been a demand to be more accurate
than that.

- Allow synchronous_standby_names to be disabled after communication
failure with all synchronous standby servers exceeds some timeout
Controversial

- Adjust rounding behavior for numeric GUC values
Controversial

*Tablespaces

- Allow WAL replay of CREATE TABLESPACE to work when the directory
structure on the recovery computer is different from the original
Thread quote: "part of the difficult, perhaps-not-worth doing impossible
problems"

- Allow per-tablespace quotas
This seems to point to the larger problem space of disk space monitoring,
and should probably be phrased thusly, and is a much bigger project or set
of projects.

- Allow tablespaces on RAM-based partitions for temporary objects
In the thread, what's desired is the ability to have some amount of
durability on a RAM-disk without WAL logging.

- Close race in DROP TABLESPACE on Windows
This refers to buildfarm failures from 2014.

*Statistics Collector

- Track number of WAL files ready to be archived in pg_stat_archiver
Thread quote: "pg_stat_archiver already has a column for
last_archived_wal and last_failed_wal, so you can already work out how many
files there must be between then and now"

*Point-In-Time Recovery

- Allow archive_mode to be changed without server restart
Controversial and old

*Standby server mode

- Allow pg_xlogfile_name() to be used in recovery mode
Controversial and old

- Change walsender so that it applies per-role settings
Old and possibly obsolete

--
[1]: /messages/by-id/CAFBsxsHbqMzDoGB3eAGmpcpB+7uae+LLi_G+o8HMEECM9CbQcQ@mail.gmail.com
/messages/by-id/CAFBsxsHbqMzDoGB3eAGmpcpB+7uae+LLi_G+o8HMEECM9CbQcQ@mail.gmail.com

--
John Naylor
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Andres Freund

andres@anarazel.de

over 5 years ago

In reply to: John Naylor (#1)

Re: cutting down the TODO list thread

Hi,

On 2020-10-27 15:24:35 -0400, John Naylor wrote:

As I mentioned in [1], I've volunteered to clear out the TODO list of items
that appear to be too difficult, controversial, or otherwise not worth
doing to warrant being listed there. I'll be working a few sections at a
time, and every so often I'll have a list of proposed items for removal. If
I don't hear objections, I'll remove the items after a few days while going
through the next set.

Where there's an email thread, I've skimmed a few messages to get a sense
of the community's thoughts on it. Where easily determined, I've taken age
into account, insofar as something from 2017 is going to get much more
benefit of doubt than something from 2008. I've added after each item a
phrase that sums up the reason I believe it doesn't belong anymore.
Feedback welcome, of course, although I suspect there won't be much.

- Prevent query cancel packets from being replayed by an attacker,
especially when using SSL
Old and difficult

FWIW, I don't think we should remove this. Our current solution has some
serious issues that we should address at some point.

- Allow WAL replay of CREATE TABLESPACE to work when the directory
structure on the recovery computer is different from the original
Thread quote: "part of the difficult, perhaps-not-worth doing impossible
problems"

I think we ought to do something here. Mostly because the current
situation makes it impossible to test many things on a single
system. And we have a partial solution with the tablespace mapping
files.

Greetings,

Andres Freund

Bruce Momjian

bruce@momjian.us

over 5 years ago

In reply to: John Naylor (#1)

Re: cutting down the TODO list thread

On Tue, Oct 27, 2020 at 03:24:35PM -0400, John Naylor wrote:

As I mentioned in [1], I've volunteered to clear out the TODO list of items
that appear to be too difficult, controversial, or otherwise not worth doing to
warrant being listed there. I'll be working a few sections at a time, and every
so often I'll have a list of proposed items for removal. If I don't hear
objections, I'll remove the items after a few days while going through the next
set.

Where there's an email thread, I've skimmed a few messages to get a sense of
the community's thoughts on it. Where easily determined, I've taken age into
account, insofar as something from 2017 is going to get much more benefit of
doubt than something from 2008. I've added after each item a phrase that sums
up the reason I believe it doesn't belong anymore. Feedback welcome, of course,
although I suspect there won't be much.

Thanks for working on this. It certainly needs new eyes (not mine). ;-)

I am fine reomving all the items below. I am kind of disappointed we
have these _stuck_ items, but I don't see a clear way forward, so let's
just remove them and see what requests we get for them.

---------------------------------------------------------------------------

**Administration

- Have custom variables be transaction-safe
ï¿½ Old and found to be difficult after attempting

- Allow custom variables to appear in pg_settings()
ï¿½ Old and controversial

- Implement the SQL-standard mechanism whereby REVOKE ROLE revokes only the
privilege granted by the invoking role, and not those granted by other roles
ï¿½ Old and difficult

- Prevent query cancel packets from being replayed by an attacker, especially
when using SSL
ï¿½ Old and difficult

*Configuration files

- Consider normalizing fractions in postgresql.conf, perhaps using '%'
ï¿½ At the time (2007), some gucs used an actual percentage.

- Add external tool to auto-tune some postgresql.conf parameters
ï¿½ There are already out-of-core tools that try to do this.

- Create utility to compute accurate random_page_cost value
ï¿½ Seems outdated: In the current age of SSDs and cloud environments, it's often
just set to 1.1, and there hasn't been a demand to be more accurate than that.

- Allow synchronous_standby_names to be disabled after communication failure
with all synchronous standby servers exceeds some timeout
ï¿½ Controversial

- Adjust rounding behavior for numeric GUC values
ï¿½ Controversial

*Tablespaces

- Allow WAL replay of CREATE TABLESPACE to work when the directory structure on
the recovery computer is different from the original
ï¿½ Thread quote: "part of the difficult, perhaps-not-worth doing impossible
problems"

- Allow per-tablespace quotas
ï¿½ This seems to point to the larger problem space of disk space monitoring, and
should probably be phrased thusly, and is a much bigger project or set of
projects.

- Allow tablespaces on RAM-based partitions for temporary objects
ï¿½ In the thread, what's desired is the ability to have some amount of
durability on a RAM-disk without WAL logging.

- Close race in DROP TABLESPACE on Windows
ï¿½ This refers to buildfarm failures from 2014.

*Statistics Collector

- Track number of WAL files ready to be archived in pg_stat_archiver
ï¿½ Thread quote: "pg_stat_archiver already has a column for last_archived_wal
and last_failed_wal, so you can already work out how many files there must be
between then and now"

*Point-In-Time Recovery

- Allow archive_mode to be changed without server restart
ï¿½ Controversial and old

*Standby server mode

- Allow pg_xlogfile_name() to be used in recovery mode
ï¿½ Controversial and old

- Change walsender so that it applies per-role settings
ï¿½ Old and possibly obsolete

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EnterpriseDB https://enterprisedb.com

The usefulness of a cup is in its emptiness, Bruce Lee

Bruce Momjian

bruce@momjian.us

over 5 years ago

In reply to: Bruce Momjian (#3)

Re: cutting down the TODO list thread

On Tue, Oct 27, 2020 at 03:46:14PM -0400, Bruce Momjian wrote:

On Tue, Oct 27, 2020 at 03:24:35PM -0400, John Naylor wrote:

As I mentioned in [1], I've volunteered to clear out the TODO list of items
that appear to be too difficult, controversial, or otherwise not worth doing to
warrant being listed there. I'll be working a few sections at a time, and every
so often I'll have a list of proposed items for removal. If I don't hear
objections, I'll remove the items after a few days while going through the next
set.

Where there's an email thread, I've skimmed a few messages to get a sense of
the community's thoughts on it. Where easily determined, I've taken age into
account, insofar as something from 2017 is going to get much more benefit of
doubt than something from 2008. I've added after each item a phrase that sums
up the reason I believe it doesn't belong anymore. Feedback welcome, of course,
although I suspect there won't be much.

Thanks for working on this. It certainly needs new eyes (not mine). ;-)

I am fine reomving all the items below. I am kind of disappointed we
have these _stuck_ items, but I don't see a clear way forward, so let's
just remove them and see what requests we get for them.

Do any of these limitations need to be documented before removing them
from the TODO list?

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EnterpriseDB https://enterprisedb.com

The usefulness of a cup is in its emptiness, Bruce Lee

Thomas Munro

thomas.munro@gmail.com

over 5 years ago

In reply to: Andres Freund (#2)

Re: cutting down the TODO list thread

On Wed, Oct 28, 2020 at 8:36 AM Andres Freund <andres@anarazel.de> wrote:

On 2020-10-27 15:24:35 -0400, John Naylor wrote:

- Allow WAL replay of CREATE TABLESPACE to work when the directory
structure on the recovery computer is different from the original
Thread quote: "part of the difficult, perhaps-not-worth doing impossible
problems"

I think we ought to do something here. Mostly because the current
situation makes it impossible to test many things on a single
system. And we have a partial solution with the tablespace mapping
files.

+1, we need to get something like this working so that we can write
decent replication tests. FWIW there was another little thread on the
topic, not listed there:

/messages/by-id/CALfoeisEF92F5nJ-aAcuWTvF_Aogxq_1bHLem_kVfM_tHc2mfg@mail.gmail.com