problems with "Shared Memory and Semaphores" section of docs

Started by Nathan Bossartover 1 year ago20 messages
#1Nathan Bossart
nathandbossart@gmail.com

(moving to a new thread)

On Thu, May 16, 2024 at 09:16:46PM -0500, Nathan Bossart wrote:

On Thu, May 16, 2024 at 04:37:10PM +0000, Imseih (AWS), Sami wrote:

Also, Not sure if I am mistaken here, but the "+ 5" in the existing docs
seems wrong.

If it refers to NUM_AUXILIARY_PROCS defined in
include/storage/proc.h, it should a "6"

#define NUM_AUXILIARY_PROCS 6

This is not a consequence of this patch, and can be dealt with
In a separate thread if my understanding is correct.

Ha, I think it should actually be "+ 7"! The value is calculated as

MaxConnections + autovacuum_max_workers + 1 + max_worker_processes + max_wal_senders + 6

Looking at the history, this documentation tends to be wrong quite often.
In v9.2, the checkpointer was introduced, and these formulas were not
updated. In v9.3, background worker processes were introduced, and the
formulas were still not updated. Finally, in v9.6, it was fixed in commit
597f7e3. Then, in v14, the archiver process was made an auxiliary process
(commit d75288f), making the formulas out-of-date again. And in v17, the
WAL summarizer was added.

On top of this, IIUC you actually need even more semaphores if your system
doesn't support atomics, and from a quick skim this doesn't seem to be
covered in this documentation.

A couple of other problems I noticed:

* max_wal_senders is missing from this sentence:

When using System V semaphores,
<productname>PostgreSQL</productname> uses one semaphore per allowed connection
(<xref linkend="guc-max-connections"/>), allowed autovacuum worker process
(<xref linkend="guc-autovacuum-max-workers"/>) and allowed background
process (<xref linkend="guc-max-worker-processes"/>), in sets of 16.

* AFAICT the discussion about the formulas in the paragraphs following the
table doesn't explain the reason for the constant.

* IMHO the following sentence is difficult to decipher, and I can't tell if
it actually matches the formula in the table:

The maximum number of semaphores in the system
is set by <varname>SEMMNS</varname>, which consequently must be at least
as high as <varname>max_connections</varname> plus
<varname>autovacuum_max_workers</varname> plus <varname>max_wal_senders</varname>,
plus <varname>max_worker_processes</varname>, plus one extra for each 16
allowed connections plus workers (see the formula in <xref
linkend="sysvipc-parameters"/>).

At a bare minimum, we should probably fix the obvious problems, but I
wonder if we could simplify this section a bit, too. If the exact values
are important, maybe we could introduce more GUCs like
shared_memory_size_in_huge_pages that can be consulted (instead of
requiring users to break out their calculators).

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Nathan Bossart (#1)
Re: problems with "Shared Memory and Semaphores" section of docs

Nathan Bossart <nathandbossart@gmail.com> writes:

[ many, many problems in documented formulas ]

At a bare minimum, we should probably fix the obvious problems, but I
wonder if we could simplify this section a bit, too.

Yup. "The definition of insanity is doing the same thing over and
over and expecting different results." Time to give up on documenting
these things in such detail. Anybody who really wants to know can
look at the source code.

If the exact values
are important, maybe we could introduce more GUCs like
shared_memory_size_in_huge_pages that can be consulted (instead of
requiring users to break out their calculators).

I don't especially like shared_memory_size_in_huge_pages, and I don't
want to introduce more of those. GUCs are not the right way to expose
values that you can't actually set. (Yeah, I'm guilty of some of the
existing ones like that, but it's still not a good thing.) Maybe it's
time to introduce a system view for such things? It could be really
simple, with name and value, or we could try to steal some additional
ideas such as units from pg_settings.

regards, tom lane

#3Nathan Bossart
nathandbossart@gmail.com
In reply to: Tom Lane (#2)
Re: problems with "Shared Memory and Semaphores" section of docs

On Fri, May 17, 2024 at 01:09:55PM -0400, Tom Lane wrote:

Nathan Bossart <nathandbossart@gmail.com> writes:

[ many, many problems in documented formulas ]

At a bare minimum, we should probably fix the obvious problems, but I
wonder if we could simplify this section a bit, too.

Yup. "The definition of insanity is doing the same thing over and
over and expecting different results." Time to give up on documenting
these things in such detail. Anybody who really wants to know can
look at the source code.

Cool. I'll at least fix the back-branches as-is, but I'll see about
revamping this stuff for v18.

If the exact values
are important, maybe we could introduce more GUCs like
shared_memory_size_in_huge_pages that can be consulted (instead of
requiring users to break out their calculators).

I don't especially like shared_memory_size_in_huge_pages, and I don't
want to introduce more of those. GUCs are not the right way to expose
values that you can't actually set. (Yeah, I'm guilty of some of the
existing ones like that, but it's still not a good thing.) Maybe it's
time to introduce a system view for such things? It could be really
simple, with name and value, or we could try to steal some additional
ideas such as units from pg_settings.

The advantage of the GUC is that its value could be seen before trying to
actually start the server. I don't dispute that it's not the right way to
surface this information, though.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

#4Imseih (AWS), Sami
simseih@amazon.com
In reply to: Nathan Bossart (#3)
Re: problems with "Shared Memory and Semaphores" section of docs

If the exact values
are important, maybe we could introduce more GUCs like
shared_memory_size_in_huge_pages that can be consulted (instead of
requiring users to break out their calculators).

I don't especially like shared_memory_size_in_huge_pages, and I don't
want to introduce more of those. GUCs are not the right way to expose
values that you can't actually set. (Yeah, I'm guilty of some of the
existing ones like that, but it's still not a good thing.) Maybe it's
time to introduce a system view for such things? It could be really
simple, with name and value, or we could try to steal some additional
ideas such as units from pg_settings.

I always found some of the preset GUCs [1]https://www.postgresql.org/docs/current/runtime-config-preset.html to be useful for writing SQLs used by
DBAs, particularly block_size, wal_block_size, server_version and server_version_num.

The advantage of the GUC is that its value could be seen before trying to
actually start the server.

Only if they have a sample in postgresql.conf file, right?
A GUC like shared_memory_size_in_huge_pages will not be.

[1]: https://www.postgresql.org/docs/current/runtime-config-preset.html

Regards,

Sami

#5Nathan Bossart
nathandbossart@gmail.com
In reply to: Nathan Bossart (#3)
1 attachment(s)
Re: problems with "Shared Memory and Semaphores" section of docs

On Fri, May 17, 2024 at 12:48:37PM -0500, Nathan Bossart wrote:

On Fri, May 17, 2024 at 01:09:55PM -0400, Tom Lane wrote:

Nathan Bossart <nathandbossart@gmail.com> writes:

At a bare minimum, we should probably fix the obvious problems, but I
wonder if we could simplify this section a bit, too.

Yup. "The definition of insanity is doing the same thing over and
over and expecting different results." Time to give up on documenting
these things in such detail. Anybody who really wants to know can
look at the source code.

Cool. I'll at least fix the back-branches as-is, but I'll see about
revamping this stuff for v18.

Attached is probably the absolute least we should do for the back-branches.

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Attachments:

v1-0001-fix-kernel-resources-docs-on-back-branches.patchtext/x-diff; charset=us-asciiDownload
From 853104fa59bb1c219f02f71ece0d5106cb6c0588 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Fri, 17 May 2024 14:17:59 -0500
Subject: [PATCH v1 1/1] fix kernel resources docs on back-branches

---
 doc/src/sgml/runtime.sgml | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index 6047b8171d..883a849e6f 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -781,13 +781,13 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
        <row>
         <entry><varname>SEMMNI</varname></entry>
         <entry>Maximum number of semaphore identifiers (i.e., sets)</entry>
-        <entry>at least <literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 5) / 16)</literal> plus room for other applications</entry>
+        <entry>at least <literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 7) / 16)</literal> plus room for other applications</entry>
        </row>
 
        <row>
         <entry><varname>SEMMNS</varname></entry>
         <entry>Maximum number of semaphores system-wide</entry>
-        <entry><literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 5) / 16) * 17</literal> plus room for other applications</entry>
+        <entry><literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 7) / 16) * 17</literal> plus room for other applications</entry>
        </row>
 
        <row>
@@ -838,7 +838,8 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
     When using System V semaphores,
     <productname>PostgreSQL</productname> uses one semaphore per allowed connection
     (<xref linkend="guc-max-connections"/>), allowed autovacuum worker process
-    (<xref linkend="guc-autovacuum-max-workers"/>) and allowed background
+    (<xref linkend="guc-autovacuum-max-workers"/>), allowed WAL sender process
+    (<xref linkend="guc-max-wal-senders"/>), and allowed background
     process (<xref linkend="guc-max-worker-processes"/>), in sets of 16.
     Each such set will
     also contain a 17th semaphore which contains a <quote>magic
@@ -852,7 +853,7 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
     linkend="sysvipc-parameters"/>).  The parameter <varname>SEMMNI</varname>
     determines the limit on the number of semaphore sets that can
     exist on the system at one time.  Hence this parameter must be at
-    least <literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 5) / 16)</literal>.
+    least <literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 7) / 16)</literal>.
     Lowering the number
     of allowed connections is a temporary workaround for failures,
     which are usually confusingly worded <quote>No space
-- 
2.25.1

#6Nathan Bossart
nathandbossart@gmail.com
In reply to: Imseih (AWS), Sami (#4)
Re: problems with "Shared Memory and Semaphores" section of docs

On Fri, May 17, 2024 at 06:30:08PM +0000, Imseih (AWS), Sami wrote:

The advantage of the GUC is that its value could be seen before trying to
actually start the server.

Only if they have a sample in postgresql.conf file, right?
A GUC like shared_memory_size_in_huge_pages will not be.

shared_memory_size_in_huge_pages is computed at runtime and can be viewed
with "postgres -C" before actually trying to start the server [0]https://www.postgresql.org/docs/devel/kernel-resources.html#LINUX-HUGE-PAGES.

[0]: https://www.postgresql.org/docs/devel/kernel-resources.html#LINUX-HUGE-PAGES

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

#7Andres Freund
andres@anarazel.de
In reply to: Imseih (AWS), Sami (#4)
Re: problems with "Shared Memory and Semaphores" section of docs

Hi,

On 2024-05-17 18:30:08 +0000, Imseih (AWS), Sami wrote:

The advantage of the GUC is that its value could be seen before trying to
actually start the server.

Only if they have a sample in postgresql.conf file, right?
A GUC like shared_memory_size_in_huge_pages will not be.

You can query gucs with -C. E.g.

postgres -D pgdev-dev -c shared_buffers=16MB -C shared_memory_size_in_huge_pages
13
postgres -D pgdev-dev -c shared_buffers=16MB -c huge_page_size=1GB -C shared_memory_size_in_huge_pages
1

Which is very useful to be able to actually configure that number of huge
pages. I don't think a system view or such would not help here.

Greetings,

Andres Freund

#8Imseih (AWS), Sami
simseih@amazon.com
In reply to: Andres Freund (#7)
Re: problems with "Shared Memory and Semaphores" section of docs

postgres -D pgdev-dev -c shared_buffers=16MB -C shared_memory_size_in_huge_pages
13
postgres -D pgdev-dev -c shared_buffers=16MB -c huge_page_size=1GB -C shared_memory_size_in_huge_pages
1

Which is very useful to be able to actually configure that number of huge
pages. I don't think a system view or such would not help here.

Oops. Totally missed the -C flag. Thanks for clarifying!

Regards,

Sami

#9Nathan Bossart
nathandbossart@gmail.com
In reply to: Nathan Bossart (#5)
1 attachment(s)
Re: problems with "Shared Memory and Semaphores" section of docs

On Fri, May 17, 2024 at 02:21:23PM -0500, Nathan Bossart wrote:

On Fri, May 17, 2024 at 12:48:37PM -0500, Nathan Bossart wrote:

Cool. I'll at least fix the back-branches as-is, but I'll see about
revamping this stuff for v18.

Attached is probably the absolute least we should do for the back-branches.

Any concerns with doing something like this [0]/messages/by-id/attachment/160360/v1-0001-fix-kernel-resources-docs-on-back-branches.patch for the back-branches? The
constant would be 6 instead of 7 on v14 through v16.

I wrote a quick sketch for what a runtime-computed GUC might look like for
v18. We don't have agreement on this approach, but I figured I'd post
something while we search for a better one.

[0]: /messages/by-id/attachment/160360/v1-0001-fix-kernel-resources-docs-on-back-branches.patch

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Attachments:

v1-0001-add-semaphores_required-GUC.patchtext/plain; charset=us-asciiDownload
From f5abd8150c317a0eaf4fb2aff79164ce929d2d96 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Tue, 21 May 2024 14:02:22 -0500
Subject: [PATCH v1 1/1] add semaphores_required GUC

---
 doc/src/sgml/config.sgml            | 14 ++++++++++++++
 doc/src/sgml/runtime.sgml           | 28 +++++++++++++++++-----------
 src/backend/storage/ipc/ipci.c      |  6 +++++-
 src/backend/utils/misc/guc_tables.c | 12 ++++++++++++
 4 files changed, 48 insertions(+), 12 deletions(-)

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 698169afdb..f6afc941df 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -11215,6 +11215,20 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
       </listitem>
      </varlistentry>
 
+     <varlistentry id="guc-semaphores-required" xreflabel="semaphores_required">
+      <term><varname>semaphores_required</varname> (<type>integer</type>)
+      <indexterm>
+       <primary><varname>semaphores_required</varname> configuration parameter</primary>
+      </indexterm>
+      </term>
+      <listitem>
+       <para>
+        Reports the number of semaphores that are needed for the server based
+        on the number of allowed connections, worker processes, etc.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="guc-ssl-library" xreflabel="ssl_library">
       <term><varname>ssl_library</varname> (<type>string</type>)
       <indexterm>
diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index 6047b8171d..e5ebfb8a8b 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -781,13 +781,13 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
        <row>
         <entry><varname>SEMMNI</varname></entry>
         <entry>Maximum number of semaphore identifiers (i.e., sets)</entry>
-        <entry>at least <literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 5) / 16)</literal> plus room for other applications</entry>
+        <entry>at least <literal>ceil(semaphores_required / 16)</literal> plus room for other applications</entry>
        </row>
 
        <row>
         <entry><varname>SEMMNS</varname></entry>
         <entry>Maximum number of semaphores system-wide</entry>
-        <entry><literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 5) / 16) * 17</literal> plus room for other applications</entry>
+        <entry><literal>ceil(semaphores_required / 16) * 17</literal> plus room for other applications</entry>
        </row>
 
        <row>
@@ -836,27 +836,33 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
 
    <para>
     When using System V semaphores,
-    <productname>PostgreSQL</productname> uses one semaphore per allowed connection
-    (<xref linkend="guc-max-connections"/>), allowed autovacuum worker process
-    (<xref linkend="guc-autovacuum-max-workers"/>) and allowed background
-    process (<xref linkend="guc-max-worker-processes"/>), in sets of 16.
+    <productname>PostgreSQL</productname> uses one semaphore per allowed connection,
+    worker process, etc., in sets of 16.
     Each such set will
     also contain a 17th semaphore which contains a <quote>magic
     number</quote>, to detect collision with semaphore sets used by
     other applications. The maximum number of semaphores in the system
     is set by <varname>SEMMNS</varname>, which consequently must be at least
-    as high as <varname>max_connections</varname> plus
-    <varname>autovacuum_max_workers</varname> plus <varname>max_wal_senders</varname>,
-    plus <varname>max_worker_processes</varname>, plus one extra for each 16
-    allowed connections plus workers (see the formula in <xref
+    as high as <xref linkend="guc-semaphores-required"/> plus one extra for
+    each set of 16 required semaphores (see the formula in <xref
     linkend="sysvipc-parameters"/>).  The parameter <varname>SEMMNI</varname>
     determines the limit on the number of semaphore sets that can
     exist on the system at one time.  Hence this parameter must be at
-    least <literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 5) / 16)</literal>.
+    least <literal>ceil(semaphores_required / 16)</literal>.
     Lowering the number
     of allowed connections is a temporary workaround for failures,
     which are usually confusingly worded <quote>No space
     left on device</quote>, from the function <function>semget</function>.
+    The number of semaphores required by <productname>PostgreSQL</productname>
+    is provided by the runtime-computed parameter
+    <varname>semaphores_required</varname>, which can be determined before
+    starting the server with a <command>postgres</command> command like:
+<programlisting>
+$ <userinput>postgres -D $PGDATA -C semaphores_required</userinput>
+</programlisting>
+    The value of <varname>semaphores_required</varname> should be input into
+    the aforementioned formulas to determine appropriate values for
+    <varname>SEMMNI</varname> and <varname>SEMMNS</varname>.
    </para>
 
    <para>
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 521ed5418c..3e030accac 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -372,11 +372,12 @@ InitializeShmemGUCs(void)
 	Size		size_b;
 	Size		size_mb;
 	Size		hp_size;
+	int			num_semas;
 
 	/*
 	 * Calculate the shared memory size and round up to the nearest megabyte.
 	 */
-	size_b = CalculateShmemSize(NULL);
+	size_b = CalculateShmemSize(&num_semas);
 	size_mb = add_size(size_b, (1024 * 1024) - 1) / (1024 * 1024);
 	sprintf(buf, "%zu", size_mb);
 	SetConfigOption("shared_memory_size", buf,
@@ -395,4 +396,7 @@ InitializeShmemGUCs(void)
 		SetConfigOption("shared_memory_size_in_huge_pages", buf,
 						PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
 	}
+
+	sprintf(buf, "%d", num_semas);
+	SetConfigOption("semaphores_required", buf, PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
 }
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 46c258be28..86157394d8 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -599,6 +599,7 @@ static int	segment_size;
 static int	shared_memory_size_mb;
 static int	shared_memory_size_in_huge_pages;
 static int	wal_block_size;
+static int	semaphores_required;
 static bool data_checksums;
 static bool integer_datetimes;
 
@@ -2291,6 +2292,17 @@ struct config_int ConfigureNamesInt[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"semaphores_required", PGC_INTERNAL, PRESET_OPTIONS,
+			gettext_noop("Shows the number of semaphores required for the server."),
+			NULL,
+			GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_RUNTIME_COMPUTED
+		},
+		&semaphores_required,
+		0, 0, INT_MAX,
+		NULL, NULL, NULL
+	},
+
 	{
 		{"commit_timestamp_buffers", PGC_POSTMASTER, RESOURCES_MEM,
 			gettext_noop("Sets the size of the dedicated buffer pool used for the commit timestamp cache."),
-- 
2.39.3 (Apple Git-146)

#10Imseih (AWS), Sami
simseih@amazon.com
In reply to: Nathan Bossart (#9)
Re: problems with "Shared Memory and Semaphores" section of docs
#11Nathan Bossart
nathandbossart@gmail.com
In reply to: Imseih (AWS), Sami (#10)
Re: problems with "Shared Memory and Semaphores" section of docs

On Tue, May 21, 2024 at 11:15:14PM +0000, Imseih (AWS), Sami wrote:

Any concerns with doing something like this [0] for the back-branches? The
constant would be 6 instead of 7 on v14 through v16.

As far as backpatching the present inconsistencies in the docs,
[0] looks good to me.

Committed.

--
nathan

#12Nathan Bossart
nathandbossart@gmail.com
In reply to: Nathan Bossart (#11)
1 attachment(s)
Re: problems with "Shared Memory and Semaphores" section of docs

On Mon, Jun 03, 2024 at 12:18:21PM -0500, Nathan Bossart wrote:

On Tue, May 21, 2024 at 11:15:14PM +0000, Imseih (AWS), Sami wrote:

As far as backpatching the present inconsistencies in the docs,
[0] looks good to me.

Committed.

Of course, as soon as I committed this, I noticed another missing reference
to max_wal_senders in the paragraph about POSIX semaphores. I plan to
commit/back-patch the attached patch within the next couple days.

--
nathan

Attachments:

add_missing_max_wal_sender_reference.patchtext/plain; charset=us-asciiDownload
diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index 883a849e6f..2f7c618886 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -884,7 +884,8 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
     When using POSIX semaphores, the number of semaphores needed is the
     same as for System V, that is one semaphore per allowed connection
     (<xref linkend="guc-max-connections"/>), allowed autovacuum worker process
-    (<xref linkend="guc-autovacuum-max-workers"/>) and allowed background
+    (<xref linkend="guc-autovacuum-max-workers"/>), allowed WAL sender process
+    (<xref linkend="guc-max-wal-senders"/>), and allowed background
     process (<xref linkend="guc-max-worker-processes"/>).
     On the platforms where this option is preferred, there is no specific
     kernel limit on the number of POSIX semaphores.
#13Nathan Bossart
nathandbossart@gmail.com
In reply to: Nathan Bossart (#12)
Re: problems with "Shared Memory and Semaphores" section of docs

On Mon, Jun 03, 2024 at 02:04:19PM -0500, Nathan Bossart wrote:

Of course, as soon as I committed this, I noticed another missing reference
to max_wal_senders in the paragraph about POSIX semaphores. I plan to
commit/back-patch the attached patch within the next couple days.

Committed.

--
nathan

#14Nathan Bossart
nathandbossart@gmail.com
In reply to: Nathan Bossart (#13)
1 attachment(s)
Re: problems with "Shared Memory and Semaphores" section of docs

Here is a rebased version of the patch for v18 that adds a runtime-computed
GUC. As I noted earlier, there still isn't a consensus on this approach.

--
nathan

Attachments:

v2-0001-add-semaphores_required-GUC.patchtext/plain; charset=us-asciiDownload
From 74f638f7df9c51f5ab47b282bb7107c4ba6cb5b6 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Tue, 21 May 2024 14:02:22 -0500
Subject: [PATCH v2 1/1] add semaphores_required GUC

---
 doc/src/sgml/config.sgml            | 14 +++++++++++
 doc/src/sgml/runtime.sgml           | 39 ++++++++++++++++-------------
 src/backend/storage/ipc/ipci.c      |  6 ++++-
 src/backend/utils/misc/guc_tables.c | 12 +++++++++
 4 files changed, 53 insertions(+), 18 deletions(-)

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 698169afdb..f6afc941df 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -11215,6 +11215,20 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
       </listitem>
      </varlistentry>
 
+     <varlistentry id="guc-semaphores-required" xreflabel="semaphores_required">
+      <term><varname>semaphores_required</varname> (<type>integer</type>)
+      <indexterm>
+       <primary><varname>semaphores_required</varname> configuration parameter</primary>
+      </indexterm>
+      </term>
+      <listitem>
+       <para>
+        Reports the number of semaphores that are needed for the server based
+        on the number of allowed connections, worker processes, etc.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="guc-ssl-library" xreflabel="ssl_library">
       <term><varname>ssl_library</varname> (<type>string</type>)
       <indexterm>
diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index 2f7c618886..c387f302d7 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -781,13 +781,13 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
        <row>
         <entry><varname>SEMMNI</varname></entry>
         <entry>Maximum number of semaphore identifiers (i.e., sets)</entry>
-        <entry>at least <literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 7) / 16)</literal> plus room for other applications</entry>
+        <entry>at least <literal>ceil(semaphores_required / 16)</literal> plus room for other applications</entry>
        </row>
 
        <row>
         <entry><varname>SEMMNS</varname></entry>
         <entry>Maximum number of semaphores system-wide</entry>
-        <entry><literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 7) / 16) * 17</literal> plus room for other applications</entry>
+        <entry><literal>ceil(semaphores_required / 16) * 17</literal> plus room for other applications</entry>
        </row>
 
        <row>
@@ -836,30 +836,38 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
 
    <para>
     When using System V semaphores,
-    <productname>PostgreSQL</productname> uses one semaphore per allowed connection
-    (<xref linkend="guc-max-connections"/>), allowed autovacuum worker process
-    (<xref linkend="guc-autovacuum-max-workers"/>), allowed WAL sender process
-    (<xref linkend="guc-max-wal-senders"/>), and allowed background
-    process (<xref linkend="guc-max-worker-processes"/>), in sets of 16.
+    <productname>PostgreSQL</productname> uses one semaphore per allowed connection,
+    worker process, etc., in sets of 16.
     Each such set will
     also contain a 17th semaphore which contains a <quote>magic
     number</quote>, to detect collision with semaphore sets used by
     other applications. The maximum number of semaphores in the system
     is set by <varname>SEMMNS</varname>, which consequently must be at least
-    as high as <varname>max_connections</varname> plus
-    <varname>autovacuum_max_workers</varname> plus <varname>max_wal_senders</varname>,
-    plus <varname>max_worker_processes</varname>, plus one extra for each 16
-    allowed connections plus workers (see the formula in <xref
+    as high as <xref linkend="guc-semaphores-required"/> plus one extra for
+    each set of 16 required semaphores (see the formula in <xref
     linkend="sysvipc-parameters"/>).  The parameter <varname>SEMMNI</varname>
     determines the limit on the number of semaphore sets that can
     exist on the system at one time.  Hence this parameter must be at
-    least <literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 7) / 16)</literal>.
+    least <literal>ceil(semaphores_required / 16)</literal>.
     Lowering the number
     of allowed connections is a temporary workaround for failures,
     which are usually confusingly worded <quote>No space
     left on device</quote>, from the function <function>semget</function>.
    </para>
 
+   <para>
+    The number of semaphores required by <productname>PostgreSQL</productname>
+    is provided by the runtime-computed parameter
+    <varname>semaphores_required</varname>, which can be determined before
+    starting the server with a <command>postgres</command> command like:
+<programlisting>
+$ <userinput>postgres -D $PGDATA -C semaphores_required</userinput>
+</programlisting>
+    The value of <varname>semaphores_required</varname> should be input into
+    the aforementioned formulas to determine appropriate values for
+    <varname>SEMMNI</varname> and <varname>SEMMNS</varname>.
+   </para>
+
    <para>
     In some cases it might also be necessary to increase
     <varname>SEMMAP</varname> to be at least on the order of
@@ -882,11 +890,8 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
 
    <para>
     When using POSIX semaphores, the number of semaphores needed is the
-    same as for System V, that is one semaphore per allowed connection
-    (<xref linkend="guc-max-connections"/>), allowed autovacuum worker process
-    (<xref linkend="guc-autovacuum-max-workers"/>), allowed WAL sender process
-    (<xref linkend="guc-max-wal-senders"/>), and allowed background
-    process (<xref linkend="guc-max-worker-processes"/>).
+    same as for System V, that is one semaphore per allowed connection,
+    worker process, etc.
     On the platforms where this option is preferred, there is no specific
     kernel limit on the number of POSIX semaphores.
    </para>
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 521ed5418c..3e030accac 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -372,11 +372,12 @@ InitializeShmemGUCs(void)
 	Size		size_b;
 	Size		size_mb;
 	Size		hp_size;
+	int			num_semas;
 
 	/*
 	 * Calculate the shared memory size and round up to the nearest megabyte.
 	 */
-	size_b = CalculateShmemSize(NULL);
+	size_b = CalculateShmemSize(&num_semas);
 	size_mb = add_size(size_b, (1024 * 1024) - 1) / (1024 * 1024);
 	sprintf(buf, "%zu", size_mb);
 	SetConfigOption("shared_memory_size", buf,
@@ -395,4 +396,7 @@ InitializeShmemGUCs(void)
 		SetConfigOption("shared_memory_size_in_huge_pages", buf,
 						PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
 	}
+
+	sprintf(buf, "%d", num_semas);
+	SetConfigOption("semaphores_required", buf, PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
 }
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 46c258be28..86157394d8 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -599,6 +599,7 @@ static int	segment_size;
 static int	shared_memory_size_mb;
 static int	shared_memory_size_in_huge_pages;
 static int	wal_block_size;
+static int	semaphores_required;
 static bool data_checksums;
 static bool integer_datetimes;
 
@@ -2291,6 +2292,17 @@ struct config_int ConfigureNamesInt[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"semaphores_required", PGC_INTERNAL, PRESET_OPTIONS,
+			gettext_noop("Shows the number of semaphores required for the server."),
+			NULL,
+			GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_RUNTIME_COMPUTED
+		},
+		&semaphores_required,
+		0, 0, INT_MAX,
+		NULL, NULL, NULL
+	},
+
 	{
 		{"commit_timestamp_buffers", PGC_POSTMASTER, RESOURCES_MEM,
 			gettext_noop("Sets the size of the dedicated buffer pool used for the commit timestamp cache."),
-- 
2.39.3 (Apple Git-146)

#15Robert Haas
robertmhaas@gmail.com
In reply to: Nathan Bossart (#14)
Re: problems with "Shared Memory and Semaphores" section of docs

On Thu, Jun 6, 2024 at 3:21 PM Nathan Bossart <nathandbossart@gmail.com> wrote:

Here is a rebased version of the patch for v18 that adds a runtime-computed
GUC. As I noted earlier, there still isn't a consensus on this approach.

I don't really like making this a GUC, but what's the other option?
It's reasonable for people to want to ask the server how many
resources it will need to start, and -C is the only tool we have for
that right now. So I feel like this is a fair thing to do.

I do think the name could use some more thought, though.
semaphores_required would end up being the same kind of thing as
shared_memory_size_in_huge_pages, but the names seem randomly
different. If semaphores_required is right here, why isn't
shared_memory_required used there? Seems more like we ought to call
this semaphores or os_semaphores or num_semaphores or
num_os_semaphores or something.

--
Robert Haas
EDB: http://www.enterprisedb.com

#16Nathan Bossart
nathandbossart@gmail.com
In reply to: Robert Haas (#15)
Re: problems with "Shared Memory and Semaphores" section of docs

On Thu, Jun 06, 2024 at 03:31:53PM -0400, Robert Haas wrote:

I don't really like making this a GUC, but what's the other option?
It's reasonable for people to want to ask the server how many
resources it will need to start, and -C is the only tool we have for
that right now. So I feel like this is a fair thing to do.

Yeah, this is how I feel, too.

I do think the name could use some more thought, though.
semaphores_required would end up being the same kind of thing as
shared_memory_size_in_huge_pages, but the names seem randomly
different. If semaphores_required is right here, why isn't
shared_memory_required used there? Seems more like we ought to call
this semaphores or os_semaphores or num_semaphores or
num_os_semaphores or something.

I'm fine with any of your suggestions. If I _had_ to pick one, I'd
probably choose num_os_semaphores because it's the most descriptive.

--
nathan

#17Nathan Bossart
nathandbossart@gmail.com
In reply to: Nathan Bossart (#16)
1 attachment(s)
Re: problems with "Shared Memory and Semaphores" section of docs

On Thu, Jun 06, 2024 at 02:51:42PM -0500, Nathan Bossart wrote:

On Thu, Jun 06, 2024 at 03:31:53PM -0400, Robert Haas wrote:

I do think the name could use some more thought, though.
semaphores_required would end up being the same kind of thing as
shared_memory_size_in_huge_pages, but the names seem randomly
different. If semaphores_required is right here, why isn't
shared_memory_required used there? Seems more like we ought to call
this semaphores or os_semaphores or num_semaphores or
num_os_semaphores or something.

I'm fine with any of your suggestions. If I _had_ to pick one, I'd
probably choose num_os_semaphores because it's the most descriptive.

Here's a new version of the patch with the GUC renamed to
num_os_semaphores.

--
nathan

Attachments:

v3-0001-add-num_os_semaphores-GUC.patchtext/plain; charset=us-asciiDownload
From 4321c19071eec065d8712ae0080d237d56a1478e Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Tue, 21 May 2024 14:02:22 -0500
Subject: [PATCH v3 1/1] add num_os_semaphores GUC

---
 doc/src/sgml/config.sgml            | 14 +++++++++++
 doc/src/sgml/runtime.sgml           | 39 ++++++++++++++++-------------
 src/backend/storage/ipc/ipci.c      |  6 ++++-
 src/backend/utils/misc/guc_tables.c | 12 +++++++++
 4 files changed, 53 insertions(+), 18 deletions(-)

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 698169afdb..a8372eadf5 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -11215,6 +11215,20 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
       </listitem>
      </varlistentry>
 
+     <varlistentry id="guc-num-os-semaphores" xreflabel="num_os_semaphores">
+      <term><varname>num_os_semaphores</varname> (<type>integer</type>)
+      <indexterm>
+       <primary><varname>num_os_semaphores</varname> configuration parameter</primary>
+      </indexterm>
+      </term>
+      <listitem>
+       <para>
+        Reports the number of semaphores that are needed for the server based
+        on the number of allowed connections, worker processes, etc.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="guc-ssl-library" xreflabel="ssl_library">
       <term><varname>ssl_library</varname> (<type>string</type>)
       <indexterm>
diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index 2f7c618886..26f3cbe555 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -781,13 +781,13 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
        <row>
         <entry><varname>SEMMNI</varname></entry>
         <entry>Maximum number of semaphore identifiers (i.e., sets)</entry>
-        <entry>at least <literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 7) / 16)</literal> plus room for other applications</entry>
+        <entry>at least <literal>ceil(num_os_semaphores / 16)</literal> plus room for other applications</entry>
        </row>
 
        <row>
         <entry><varname>SEMMNS</varname></entry>
         <entry>Maximum number of semaphores system-wide</entry>
-        <entry><literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 7) / 16) * 17</literal> plus room for other applications</entry>
+        <entry><literal>ceil(num_os_semaphores / 16) * 17</literal> plus room for other applications</entry>
        </row>
 
        <row>
@@ -836,30 +836,38 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
 
    <para>
     When using System V semaphores,
-    <productname>PostgreSQL</productname> uses one semaphore per allowed connection
-    (<xref linkend="guc-max-connections"/>), allowed autovacuum worker process
-    (<xref linkend="guc-autovacuum-max-workers"/>), allowed WAL sender process
-    (<xref linkend="guc-max-wal-senders"/>), and allowed background
-    process (<xref linkend="guc-max-worker-processes"/>), in sets of 16.
+    <productname>PostgreSQL</productname> uses one semaphore per allowed connection,
+    worker process, etc., in sets of 16.
     Each such set will
     also contain a 17th semaphore which contains a <quote>magic
     number</quote>, to detect collision with semaphore sets used by
     other applications. The maximum number of semaphores in the system
     is set by <varname>SEMMNS</varname>, which consequently must be at least
-    as high as <varname>max_connections</varname> plus
-    <varname>autovacuum_max_workers</varname> plus <varname>max_wal_senders</varname>,
-    plus <varname>max_worker_processes</varname>, plus one extra for each 16
-    allowed connections plus workers (see the formula in <xref
+    as high as <xref linkend="guc-num-os-semaphores"/> plus one extra for
+    each set of 16 required semaphores (see the formula in <xref
     linkend="sysvipc-parameters"/>).  The parameter <varname>SEMMNI</varname>
     determines the limit on the number of semaphore sets that can
     exist on the system at one time.  Hence this parameter must be at
-    least <literal>ceil((max_connections + autovacuum_max_workers + max_wal_senders + max_worker_processes + 7) / 16)</literal>.
+    least <literal>ceil(num_os_semaphores / 16)</literal>.
     Lowering the number
     of allowed connections is a temporary workaround for failures,
     which are usually confusingly worded <quote>No space
     left on device</quote>, from the function <function>semget</function>.
    </para>
 
+   <para>
+    The number of semaphores required by <productname>PostgreSQL</productname>
+    is provided by the runtime-computed parameter
+    <varname>num_os_semaphores</varname>, which can be determined before
+    starting the server with a <command>postgres</command> command like:
+<programlisting>
+$ <userinput>postgres -D $PGDATA -C num_os_semaphores</userinput>
+</programlisting>
+    The value of <varname>num_os_semaphores</varname> should be input into
+    the aforementioned formulas to determine appropriate values for
+    <varname>SEMMNI</varname> and <varname>SEMMNS</varname>.
+   </para>
+
    <para>
     In some cases it might also be necessary to increase
     <varname>SEMMAP</varname> to be at least on the order of
@@ -882,11 +890,8 @@ psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such
 
    <para>
     When using POSIX semaphores, the number of semaphores needed is the
-    same as for System V, that is one semaphore per allowed connection
-    (<xref linkend="guc-max-connections"/>), allowed autovacuum worker process
-    (<xref linkend="guc-autovacuum-max-workers"/>), allowed WAL sender process
-    (<xref linkend="guc-max-wal-senders"/>), and allowed background
-    process (<xref linkend="guc-max-worker-processes"/>).
+    same as for System V, that is one semaphore per allowed connection,
+    worker process, etc.
     On the platforms where this option is preferred, there is no specific
     kernel limit on the number of POSIX semaphores.
    </para>
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 521ed5418c..0314513aa6 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -372,11 +372,12 @@ InitializeShmemGUCs(void)
 	Size		size_b;
 	Size		size_mb;
 	Size		hp_size;
+	int			num_semas;
 
 	/*
 	 * Calculate the shared memory size and round up to the nearest megabyte.
 	 */
-	size_b = CalculateShmemSize(NULL);
+	size_b = CalculateShmemSize(&num_semas);
 	size_mb = add_size(size_b, (1024 * 1024) - 1) / (1024 * 1024);
 	sprintf(buf, "%zu", size_mb);
 	SetConfigOption("shared_memory_size", buf,
@@ -395,4 +396,7 @@ InitializeShmemGUCs(void)
 		SetConfigOption("shared_memory_size_in_huge_pages", buf,
 						PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
 	}
+
+	sprintf(buf, "%d", num_semas);
+	SetConfigOption("num_os_semaphores", buf, PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
 }
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 46c258be28..80e77cbac9 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -599,6 +599,7 @@ static int	segment_size;
 static int	shared_memory_size_mb;
 static int	shared_memory_size_in_huge_pages;
 static int	wal_block_size;
+static int	num_os_semaphores;
 static bool data_checksums;
 static bool integer_datetimes;
 
@@ -2291,6 +2292,17 @@ struct config_int ConfigureNamesInt[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"num_os_semaphores", PGC_INTERNAL, PRESET_OPTIONS,
+			gettext_noop("Shows the number of semaphores required for the server."),
+			NULL,
+			GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_RUNTIME_COMPUTED
+		},
+		&num_os_semaphores,
+		0, 0, INT_MAX,
+		NULL, NULL, NULL
+	},
+
 	{
 		{"commit_timestamp_buffers", PGC_POSTMASTER, RESOURCES_MEM,
 			gettext_noop("Sets the size of the dedicated buffer pool used for the commit timestamp cache."),
-- 
2.39.3 (Apple Git-146)

#18Nathan Bossart
nathandbossart@gmail.com
In reply to: Nathan Bossart (#17)
Re: problems with "Shared Memory and Semaphores" section of docs

On Sun, Jun 09, 2024 at 02:04:17PM -0500, Nathan Bossart wrote:

Here's a new version of the patch with the GUC renamed to
num_os_semaphores.

The only thing stopping me from committing this right now is Tom's upthread
objection about adding more GUCs that just expose values that you can't
actually set. If that objection still stands, I'll withdraw this patch
(and maybe try introducing a new way to surface this information someday).

--
nathan

#19Tom Lane
tgl@sss.pgh.pa.us
In reply to: Nathan Bossart (#18)
Re: problems with "Shared Memory and Semaphores" section of docs

Nathan Bossart <nathandbossart@gmail.com> writes:

The only thing stopping me from committing this right now is Tom's upthread
objection about adding more GUCs that just expose values that you can't
actually set. If that objection still stands, I'll withdraw this patch
(and maybe try introducing a new way to surface this information someday).

It still feels to me like not a great way to go about it. Having
said that, it's not like we don't have any existing examples of
the category, so I won't cry hard if I'm outvoted.

regards, tom lane

#20Nathan Bossart
nathandbossart@gmail.com
In reply to: Tom Lane (#19)
Re: problems with "Shared Memory and Semaphores" section of docs

Committed.

--
nathan