postgresql.conf.sample ordering for IO, worker related GUCs
Hi,
While working on polishing the AIO patchset, I was trying to figure out where
to fit the new GUCs. So far I had a new "top-level" #--- style section named
"WIP AIO GUC docs" which I suspect you all won't let me get away with.
There is an existing (sub-)section which already has a few related GUCs and
could fit AIO related ones.
#------------------------------------------------------------------------------
# RESOURCE USAGE (except WAL)
#------------------------------------------------------------------------------
...
# - Asynchronous Behavior -
Except that the list of GUCs in it seems confusing:
#backend_flush_after = 0 # measured in pages, 0 disables
#effective_io_concurrency = 1 # 1-1000; 0 disables prefetching
#maintenance_io_concurrency = 10 # 1-1000; 0 disables prefetching
#io_combine_limit = 128kB # usually 1-32 blocks (depends on OS)
#max_worker_processes = 8 # (change requires restart)
#max_parallel_workers_per_gather = 2 # limited by max_parallel_workers
#max_parallel_maintenance_workers = 2 # limited by max_parallel_workers
#max_parallel_workers = 8 # number of max_worker_processes that
# can be used in parallel operations
#parallel_leader_participation = on
The first four GUCs make some sense under the heading, but why are the worker
GUCs in the same section?
Seems like it would make sense to introduce a new "Worker Processes"
sub-section?
I am wondering if it'd be better to get rid of "Asynchronous Behavior" and
instead have "# - IO -" or such? E.g. io_combine_limit isn't really about
anything asynchronous, but still seems like it should belong in the same
section as the rest?
I also wonder if fsync, recovery_init_sync_method, data_sync_retry shouldn't
be in that new subsection. Or maybe it should even be a top-level section.
FWIW, AIO currently adds the following GUCs:
#io_method = worker # worker, io_uring, sync
# (change requires restart)
#io_max_concurrency = -1 # Max number of IOs that may be in
# flight at the same time in one backend
# (change requires restart)
#io_workers = 3 # 1-32; relevant only if io_method = worker
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
While working on polishing the AIO patchset, I was trying to figure out where
to fit the new GUCs. So far I had a new "top-level" #--- style section named
"WIP AIO GUC docs" which I suspect you all won't let me get away with.
There is an existing (sub-)section which already has a few related GUCs and
could fit AIO related ones.
I think the normal theory for postgresql.conf.sample is that it should
match the organization of config.sgml. What are you planning there?
regards, tom lane
Hi
On January 30, 2025 8:55:36 PM EST, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andres Freund <andres@anarazel.de> writes:
While working on polishing the AIO patchset, I was trying to figure out where
to fit the new GUCs. So far I had a new "top-level" #--- style section named
"WIP AIO GUC docs" which I suspect you all won't let me get away with.
There is an existing (sub-)section which already has a few related GUCs and
could fit AIO related ones.I think the normal theory for postgresql.conf.sample is that it should
match the organization of config.sgml. What are you planning there?
Pretty much the same. I.e. I'm thinking that the worker stuff should be it's own subsection and that the existing IO parameters should be moved to either a new subsection or a new top level section. But I'm wondering how others think it should be structured...
Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Hi,
On 2025-01-30 21:24:05 -0500, Andres Freund wrote:
On January 30, 2025 8:55:36 PM EST, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andres Freund <andres@anarazel.de> writes:
While working on polishing the AIO patchset, I was trying to figure out where
to fit the new GUCs. So far I had a new "top-level" #--- style section named
"WIP AIO GUC docs" which I suspect you all won't let me get away with.
There is an existing (sub-)section which already has a few related GUCs and
could fit AIO related ones.I think the normal theory for postgresql.conf.sample is that it should
match the organization of config.sgml. What are you planning there?Pretty much the same. I.e. I'm thinking that the worker stuff should be it's
own subsection and that the existing IO parameters should be moved to either
a new subsection or a new top level section. But I'm wondering how others
think it should be structured...
Here are draft changes for the minimal thing I think we should do.
I don't really know what to do about the "IO" abbreviation. The other sections
un-abbreviate abbreviations, but I suspect that Input/Output will be less
informative than IO to most...
I still wonder if we instead ought to create a top-level "IO" section instead
of leaving it under "Resource Usage". How many IOs we combine, how
aggressively we flush unflushed data, etc only kinda fits into the resource
usage category.
Greetings,
Andres Freund
Attachments:
v1-0001-config-Split-Worker-Processes-out-of-Asynchronous.patchtext/x-diff; charset=us-asciiDownload
From a32a74f5285ec151086bc8cc8f6a85bde77ba323 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Fri, 31 Jan 2025 10:03:40 -0500
Subject: [PATCH v1 1/2] config: Split "Worker Processes" out of "Asynchronous
Behavior"
Discussion: https://postgr.es/m/x3tlw2jk5gm3r3mv47hwrshffyw7halpczkfbk3peksxds7bvc@lguk43z3bsyq
---
src/include/utils/guc_tables.h | 1 +
src/backend/utils/misc/guc_tables.c | 11 ++++++-----
src/backend/utils/misc/postgresql.conf.sample | 3 +++
doc/src/sgml/config.sgml | 6 ++++++
4 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index 3f18ff451ef..0b16b4dcf57 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -65,6 +65,7 @@ enum config_group
RESOURCES_KERNEL,
RESOURCES_BGWRITER,
RESOURCES_ASYNCHRONOUS,
+ RESOURCES_WORKER_PROCESSES,
WAL_SETTINGS,
WAL_CHECKPOINTS,
WAL_ARCHIVING,
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 38cb9e970d5..c6a999a64cd 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -683,6 +683,7 @@ const char *const config_group_names[] =
[RESOURCES_KERNEL] = gettext_noop("Resource Usage / Kernel Resources"),
[RESOURCES_BGWRITER] = gettext_noop("Resource Usage / Background Writer"),
[RESOURCES_ASYNCHRONOUS] = gettext_noop("Resource Usage / Asynchronous Behavior"),
+ [RESOURCES_WORKER_PROCESSES] = gettext_noop("Resource Usage / Worker Processes"),
[WAL_SETTINGS] = gettext_noop("Write-Ahead Log / Settings"),
[WAL_CHECKPOINTS] = gettext_noop("Write-Ahead Log / Checkpoints"),
[WAL_ARCHIVING] = gettext_noop("Write-Ahead Log / Archiving"),
@@ -1962,7 +1963,7 @@ struct config_bool ConfigureNamesBool[] =
},
{
- {"parallel_leader_participation", PGC_USERSET, RESOURCES_ASYNCHRONOUS,
+ {"parallel_leader_participation", PGC_USERSET, RESOURCES_WORKER_PROCESSES,
gettext_noop("Controls whether Gather and Gather Merge also run subplans."),
gettext_noop("Should gather nodes also run subplans or just gather tuples?"),
GUC_EXPLAIN
@@ -3234,7 +3235,7 @@ struct config_int ConfigureNamesInt[] =
{
{"max_worker_processes",
PGC_POSTMASTER,
- RESOURCES_ASYNCHRONOUS,
+ RESOURCES_WORKER_PROCESSES,
gettext_noop("Maximum number of concurrent worker processes."),
NULL,
},
@@ -3487,7 +3488,7 @@ struct config_int ConfigureNamesInt[] =
},
{
- {"max_parallel_maintenance_workers", PGC_USERSET, RESOURCES_ASYNCHRONOUS,
+ {"max_parallel_maintenance_workers", PGC_USERSET, RESOURCES_WORKER_PROCESSES,
gettext_noop("Sets the maximum number of parallel processes per maintenance operation."),
NULL
},
@@ -3497,7 +3498,7 @@ struct config_int ConfigureNamesInt[] =
},
{
- {"max_parallel_workers_per_gather", PGC_USERSET, RESOURCES_ASYNCHRONOUS,
+ {"max_parallel_workers_per_gather", PGC_USERSET, RESOURCES_WORKER_PROCESSES,
gettext_noop("Sets the maximum number of parallel processes per executor node."),
NULL,
GUC_EXPLAIN
@@ -3508,7 +3509,7 @@ struct config_int ConfigureNamesInt[] =
},
{
- {"max_parallel_workers", PGC_USERSET, RESOURCES_ASYNCHRONOUS,
+ {"max_parallel_workers", PGC_USERSET, RESOURCES_WORKER_PROCESSES,
gettext_noop("Sets the maximum number of parallel workers that can be active at one time."),
NULL,
GUC_EXPLAIN
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 079efa1baa7..1197efb26df 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -198,6 +198,9 @@
#effective_io_concurrency = 1 # 1-1000; 0 disables prefetching
#maintenance_io_concurrency = 10 # 1-1000; 0 disables prefetching
#io_combine_limit = 128kB # usually 1-32 blocks (depends on OS)
+
+# - Worker Processes -
+
#max_worker_processes = 8 # (change requires restart)
#max_parallel_workers_per_gather = 2 # limited by max_parallel_workers
#max_parallel_maintenance_workers = 2 # limited by max_parallel_workers
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index a782f109982..ca6077d1666 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2618,7 +2618,13 @@ include_dir 'conf.d'
</para>
</listitem>
</varlistentry>
+ </variablelist>
+ </sect2>
+ <sect2 id="runtime-config-resource-worker-processes">
+ <title>Worker Processes</title>
+
+ <variablelist>
<varlistentry id="guc-max-worker-processes" xreflabel="max_worker_processes">
<term><varname>max_worker_processes</varname> (<type>integer</type>)
<indexterm>
--
2.48.1.76.g4e746b1a31.dirty
v1-0002-config-Rename-Asynchronous-Behavior-to-IO.patchtext/x-diff; charset=us-asciiDownload
From ceb74f652917c662175f77994b53e76aed54546f Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Fri, 31 Jan 2025 10:20:21 -0500
Subject: [PATCH v1 2/2] config: Rename "Asynchronous Behavior" to "IO"
"IO" seems more descriptive than "Asynchronous Behavior", given that some of
the GUCs in the section don't relate to anything asynchronous.
Most other abbreviations in the config sections are un-abbreviated, but
"Input/Output" seems less likely to be helpful than just IO or I/O.
Discussion: https://postgr.es/m/x3tlw2jk5gm3r3mv47hwrshffyw7halpczkfbk3peksxds7bvc@lguk43z3bsyq
---
src/include/utils/guc_tables.h | 2 +-
src/backend/utils/misc/guc_tables.c | 10 +++++-----
src/backend/utils/misc/postgresql.conf.sample | 2 +-
doc/src/sgml/config.sgml | 4 ++--
4 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index 0b16b4dcf57..1c1c5b6ea52 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -64,7 +64,7 @@ enum config_group
RESOURCES_DISK,
RESOURCES_KERNEL,
RESOURCES_BGWRITER,
- RESOURCES_ASYNCHRONOUS,
+ RESOURCES_IO,
RESOURCES_WORKER_PROCESSES,
WAL_SETTINGS,
WAL_CHECKPOINTS,
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index c6a999a64cd..81d80ef1bc0 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -682,7 +682,7 @@ const char *const config_group_names[] =
[RESOURCES_DISK] = gettext_noop("Resource Usage / Disk"),
[RESOURCES_KERNEL] = gettext_noop("Resource Usage / Kernel Resources"),
[RESOURCES_BGWRITER] = gettext_noop("Resource Usage / Background Writer"),
- [RESOURCES_ASYNCHRONOUS] = gettext_noop("Resource Usage / Asynchronous Behavior"),
+ [RESOURCES_IO] = gettext_noop("Resource Usage / IO"),
[RESOURCES_WORKER_PROCESSES] = gettext_noop("Resource Usage / Worker Processes"),
[WAL_SETTINGS] = gettext_noop("Write-Ahead Log / Settings"),
[WAL_CHECKPOINTS] = gettext_noop("Write-Ahead Log / Checkpoints"),
@@ -3181,7 +3181,7 @@ struct config_int ConfigureNamesInt[] =
{
{"effective_io_concurrency",
PGC_USERSET,
- RESOURCES_ASYNCHRONOUS,
+ RESOURCES_IO,
gettext_noop("Number of simultaneous requests that can be handled efficiently by the disk subsystem."),
NULL,
GUC_EXPLAIN
@@ -3195,7 +3195,7 @@ struct config_int ConfigureNamesInt[] =
{
{"maintenance_io_concurrency",
PGC_USERSET,
- RESOURCES_ASYNCHRONOUS,
+ RESOURCES_IO,
gettext_noop("A variant of \"effective_io_concurrency\" that is used for maintenance work."),
NULL,
GUC_EXPLAIN
@@ -3210,7 +3210,7 @@ struct config_int ConfigureNamesInt[] =
{
{"io_combine_limit",
PGC_USERSET,
- RESOURCES_ASYNCHRONOUS,
+ RESOURCES_IO,
gettext_noop("Limit on the size of data reads and writes."),
NULL,
GUC_UNIT_BLOCKS
@@ -3222,7 +3222,7 @@ struct config_int ConfigureNamesInt[] =
},
{
- {"backend_flush_after", PGC_USERSET, RESOURCES_ASYNCHRONOUS,
+ {"backend_flush_after", PGC_USERSET, RESOURCES_IO,
gettext_noop("Number of pages after which previously performed writes are flushed to disk."),
NULL,
GUC_UNIT_BLOCKS
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 1197efb26df..a712ae1a375 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -192,7 +192,7 @@
#bgwriter_lru_multiplier = 2.0 # 0-10.0 multiplier on buffers scanned/round
#bgwriter_flush_after = 0 # measured in pages, 0 disables
-# - Asynchronous Behavior -
+# - IO -
#backend_flush_after = 0 # measured in pages, 0 disables
#effective_io_concurrency = 1 # 1-1000; 0 disables prefetching
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index ca6077d1666..3d1919969d9 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2506,8 +2506,8 @@ include_dir 'conf.d'
</para>
</sect2>
- <sect2 id="runtime-config-resource-async-behavior">
- <title>Asynchronous Behavior</title>
+ <sect2 id="runtime-config-resource-io">
+ <title>IO</title>
<variablelist>
<varlistentry id="guc-backend-flush-after" xreflabel="backend_flush_after">
--
2.48.1.76.g4e746b1a31.dirty
Andres Freund <andres@anarazel.de> writes:
I don't really know what to do about the "IO" abbreviation. The other sections
un-abbreviate abbreviations, but I suspect that Input/Output will be less
informative than IO to most...
I'd lean towards writing "I/O", but it's not a point I'd quibble over.
regards, tom lane
On Fri, Jan 31, 2025 at 10:25 AM Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2025-01-30 21:24:05 -0500, Andres Freund wrote:
On January 30, 2025 8:55:36 PM EST, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andres Freund <andres@anarazel.de> writes:
While working on polishing the AIO patchset, I was trying to figure out where
to fit the new GUCs. So far I had a new "top-level" #--- style section named
"WIP AIO GUC docs" which I suspect you all won't let me get away with.
There is an existing (sub-)section which already has a few related GUCs and
could fit AIO related ones.I think the normal theory for postgresql.conf.sample is that it should
match the organization of config.sgml. What are you planning there?Pretty much the same. I.e. I'm thinking that the worker stuff should be it's
own subsection and that the existing IO parameters should be moved to either
a new subsection or a new top level section. But I'm wondering how others
think it should be structured...Here are draft changes for the minimal thing I think we should do.
I don't really know what to do about the "IO" abbreviation. The other sections
un-abbreviate abbreviations, but I suspect that Input/Output will be less
informative than IO to most...I still wonder if we instead ought to create a top-level "IO" section instead
of leaving it under "Resource Usage". How many IOs we combine, how
aggressively we flush unflushed data, etc only kinda fits into the resource
usage category.
+1 from me, though I did pause on whether it should be called
"background workers" rather than "worker processes", but I think this
is the right direction.
Robert Treat
https://xzilla.net
Hi,
On January 31, 2025 5:22:43 PM EST, Robert Treat <rob@xzilla.net> wrote:
On Fri, Jan 31, 2025 at 10:25 AM Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2025-01-30 21:24:05 -0500, Andres Freund wrote:
On January 30, 2025 8:55:36 PM EST, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andres Freund <andres@anarazel.de> writes:
While working on polishing the AIO patchset, I was trying to figure out where
to fit the new GUCs. So far I had a new "top-level" #--- style section named
"WIP AIO GUC docs" which I suspect you all won't let me get away with.
There is an existing (sub-)section which already has a few related GUCs and
could fit AIO related ones.I think the normal theory for postgresql.conf.sample is that it should
match the organization of config.sgml. What are you planning there?Pretty much the same. I.e. I'm thinking that the worker stuff should be it's
own subsection and that the existing IO parameters should be moved to either
a new subsection or a new top level section. But I'm wondering how others
think it should be structured...Here are draft changes for the minimal thing I think we should do.
I don't really know what to do about the "IO" abbreviation. The other sections
un-abbreviate abbreviations, but I suspect that Input/Output will be less
informative than IO to most...I still wonder if we instead ought to create a top-level "IO" section instead
of leaving it under "Resource Usage". How many IOs we combine, how
aggressively we flush unflushed data, etc only kinda fits into the resource
usage category.+1 from me, though I did pause on whether it should be called
"background workers" rather than "worker processes", but I think this
is the right direction.
+1 for the patch as-is, or my suggestion to make it a top level section?
Greetings,
Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Hi,
On 2025-01-31 18:21:54 -0500, Andres Freund wrote:
On January 31, 2025 5:22:43 PM EST, Robert Treat <rob@xzilla.net> wrote:
On Fri, Jan 31, 2025 at 10:25 AM Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2025-01-30 21:24:05 -0500, Andres Freund wrote:
On January 30, 2025 8:55:36 PM EST, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andres Freund <andres@anarazel.de> writes:
While working on polishing the AIO patchset, I was trying to figure out where
to fit the new GUCs. So far I had a new "top-level" #--- style section named
"WIP AIO GUC docs" which I suspect you all won't let me get away with.
There is an existing (sub-)section which already has a few related GUCs and
could fit AIO related ones.I think the normal theory for postgresql.conf.sample is that it should
match the organization of config.sgml. What are you planning there?Pretty much the same. I.e. I'm thinking that the worker stuff should be it's
own subsection and that the existing IO parameters should be moved to either
a new subsection or a new top level section. But I'm wondering how others
think it should be structured...Here are draft changes for the minimal thing I think we should do.
I don't really know what to do about the "IO" abbreviation. The other sections
un-abbreviate abbreviations, but I suspect that Input/Output will be less
informative than IO to most...I still wonder if we instead ought to create a top-level "IO" section instead
of leaving it under "Resource Usage". How many IOs we combine, how
aggressively we flush unflushed data, etc only kinda fits into the resource
usage category.+1 from me, though I did pause on whether it should be called
"background workers" rather than "worker processes", but I think this
is the right direction.+1 for the patch as-is, or my suggestion to make it a top level section?
I pushed the patch as I had it, save for replacing IO with I/O where
appropriate. If we want to go for a larger restructuring in the future, this
won't make it harder, and perhaps even a little simpler.
Greetings,
Andres Freund