enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

Started by Bharath Rupireddyover 4 years ago18 messages
#1Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com

Hi,

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of these
processes. Inside the code, we could use the AuxiliaryPidGetProc() to
get the PGPROC of these processes. Note that, neither
AuxiliaryPidGetProc() nor BackendPidGetProc() can return PGPROC(as
they don't have PGPROC entries at all) entries for the syslogger,
stats collector processes.

Open points:
1) I'm not sure if it's a good idea to log postmaster memory usage
too. Thoughts?
2) Since with this change pg_log_backend_memory_contexts() will work
for auxiliary processes too, do we need to change the function name
from pg_log_backend_memory_contexts() to
pg_log_backend_memory_contexts()/pg_log_memory_contexts()/some other
name? Or is it a good idea to have a separate function for auxiliary
processes alone, pg_log_auxilliary_process_memory_contexts()?
Thoughts?

I will attach the patch, if possible with test cases, once we agree on
the above open points.

Regards,
Bharath Rupireddy.

#2torikoshia
torikoshia@oss.nttdata.com
In reply to: Bharath Rupireddy (#1)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

Thanks for working on this!

On 2021-10-09 22:23, Bharath Rupireddy wrote:

Hi,

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of these
processes.

As the discussion below, we thought logging memory contexts of other
than client backends is possible but were not sure how useful it is.
After all, we have ended up restricting the target process to client
backends for now.

/messages/by-id/0b0657d5febd0e46565a6bc9c62ba3f6@oss.nttdata.com

If we can use debuggers, it's possible to know the memory contexts e.g.
using MemoryContextStats().
So IMHO if it's necessary to know memory contexts without attaching gdb
for other than client backends(probably this means using under
production environment), this enhancement would be pay.

--
Regards,

--
Atsushi Torikoshi
NTT DATA CORPORATION

#3Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: torikoshia (#2)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Mon, Oct 11, 2021 at 8:21 AM torikoshia <torikoshia@oss.nttdata.com> wrote:

Thanks for working on this!

On 2021-10-09 22:23, Bharath Rupireddy wrote:

Hi,

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of these
processes.

As the discussion below, we thought logging memory contexts of other
than client backends is possible but were not sure how useful it is.
After all, we have ended up restricting the target process to client
backends for now.

/messages/by-id/0b0657d5febd0e46565a6bc9c62ba3f6@oss.nttdata.com

If we can use debuggers, it's possible to know the memory contexts e.g.
using MemoryContextStats().
So IMHO if it's necessary to know memory contexts without attaching gdb
for other than client backends(probably this means using under
production environment), this enhancement would be pay.

Thanks for providing your thoughts. Knowing memory usage of auxiliary
processes is as important as backends (user session processes) without
attaching debugger in production environments.

There are some open points as mentioned in my first mail in this
thread, I will start working on this patch once we agree on them.

Regards,
Bharath Rupireddy.

#4Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Bharath Rupireddy (#3)
1 attachment(s)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Mon, Oct 11, 2021 at 9:55 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Mon, Oct 11, 2021 at 8:21 AM torikoshia <torikoshia@oss.nttdata.com> wrote:

Thanks for working on this!

On 2021-10-09 22:23, Bharath Rupireddy wrote:

Hi,

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of these
processes.

As the discussion below, we thought logging memory contexts of other
than client backends is possible but were not sure how useful it is.
After all, we have ended up restricting the target process to client
backends for now.

/messages/by-id/0b0657d5febd0e46565a6bc9c62ba3f6@oss.nttdata.com

If we can use debuggers, it's possible to know the memory contexts e.g.
using MemoryContextStats().
So IMHO if it's necessary to know memory contexts without attaching gdb
for other than client backends(probably this means using under
production environment), this enhancement would be pay.

Thanks for providing your thoughts. Knowing memory usage of auxiliary
processes is as important as backends (user session processes) without
attaching debugger in production environments.

There are some open points as mentioned in my first mail in this
thread, I will start working on this patch once we agree on them.

I'm attaching the v1 patch that enables
pg_log_backend_memory_contexts() to log memory contexts of auxiliary
processes. Please review it.

Here's the CF entry - https://commitfest.postgresql.org/35/3385/

Regards,
Bharath Rupireddy.

Attachments:

v1-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchapplication/octet-stream; name=v1-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchDownload
From e5ac2524ac6781bbc552269b06e157bdb0d1db86 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 29 Oct 2021 16:42:18 +0000
Subject: [PATCH v1] enhance pg_log_backend_memory_contexts to log memory
 contexts of aux procs

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of
these processes. Inside the code, we could use the AuxiliaryPidGetProc()
to get the PGPROC of these processes. Note that, neither
AuxiliaryPidGetProc() nor BackendPidGetProc() can return PGPROC(as
they don't have PGPROC entries at all) entries for the syslogger,
stats collector processes.
---
 doc/src/sgml/func.sgml                       | 14 +++++---
 src/backend/utils/adt/mcxtfuncs.c            | 29 +++++++++++-----
 src/test/regress/expected/misc_functions.out | 35 ++++++++++++++++++++
 src/test/regress/sql/misc_functions.sql      | 13 ++++++++
 4 files changed, 79 insertions(+), 12 deletions(-)

diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 4b49dff2ff..2449434abf 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25325,10 +25325,16 @@ SELECT collation for ('foo' COLLATE "de_DE");
         <returnvalue>boolean</returnvalue>
        </para>
        <para>
-        Requests to log the memory contexts of the backend with the
-        specified process ID.  These memory contexts will be logged at
-        <literal>LOG</literal> message level. They will appear in
-        the server log based on the log configuration set
+        Requests to log memory contexts of the backend or the
+        <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or
+        the <glossterm linkend="glossary-auxiliary-proc">auxiliary process</glossterm>
+        with the specified process ID. All of the
+        <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>
+        are supported except the <glossterm linkend="glossary-logger">logger</glossterm>
+        and the <glossterm linkend="glossary-stats-collector">statistics collector</glossterm>
+        as they are not connected to shared memory the function can not make requests.
+        These memory contexts will be logged at <literal>LOG</literal> message level.
+        They will appear in the server log based on the log configuration set
         (See <xref linkend="runtime-config-logging"/> for more information),
         but will not be sent to the client regardless of
         <xref linkend="guc-client-min-messages"/>.
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..67598ddbb3 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -176,17 +176,26 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 {
 	int			pid = PG_GETARG_INT32(0);
 	PGPROC	   *proc;
+	bool		is_aux_proc = false;
+	BackendId   backendId = InvalidBackendId;
 
 	proc = BackendPidGetProc(pid);
 
+	/* See if the process with given pid is an auxiliary process. */
+	if (proc == NULL)
+	{
+		proc = AuxiliaryPidGetProc(pid);
+		is_aux_proc = true;
+	}
+
 	/*
-	 * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
-	 * we reach kill(), a process for which we get a valid proc here might
-	 * have terminated on its own.  There's no way to acquire a lock on an
-	 * arbitrary process to prevent that. But since this mechanism is usually
-	 * used to debug a backend running and consuming lots of memory, that it
-	 * might end on its own first and its memory contexts are not logged is
-	 * not a problem.
+	 * BackendPidGetProc or AuxiliaryPidGetProc returns NULL if the pid isn't
+	 * valid; but by the time we reach kill(), a process for which we get a
+	 * valid proc here might have terminated on its own.  There's no way to
+	 * acquire a lock on an arbitrary process to prevent that. But since this
+	 * mechanism is usually used to debug a backend running and consuming lots
+	 * of memory, that it might end on its own first and its memory contexts
+	 * are not logged is not a problem.
 	 */
 	if (proc == NULL)
 	{
@@ -199,7 +208,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 		PG_RETURN_BOOL(false);
 	}
 
-	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
+	/* Only regular backends will have valid backend id, auxiliary processes don't. */
+	if (!is_aux_proc)
+		backendId = proc->backendId;
+
+	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
 	{
 		/* Again, just a warning to allow loops */
 		ereport(WARNING,
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 71d316cad3..f2a0965e2d 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -147,6 +147,41 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
  t
 (1 row)
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('autovacuum launcher'));
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('logical replication launcher'));
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('background writer'));
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('walwriter'));
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
+DROP FUNCTION memcxt_get_proc_pid(text);
 CREATE ROLE regress_log_memory;
 SELECT has_function_privilege('regress_log_memory',
   'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 8c23874b3f..c87455c937 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -41,6 +41,19 @@ SELECT num_nulls();
 
 SELECT pg_log_backend_memory_contexts(pg_backend_pid());
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('autovacuum launcher'));
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('logical replication launcher'));
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('background writer'));
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('walwriter'));
+
+DROP FUNCTION memcxt_get_proc_pid(text);
+
 CREATE ROLE regress_log_memory;
 
 SELECT has_function_privilege('regress_log_memory',
-- 
2.25.1

#5Kyotaro Horiguchi
horikyota.ntt@gmail.com
In reply to: Bharath Rupireddy (#4)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

At Fri, 29 Oct 2021 22:25:04 +0530, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote in

On Mon, Oct 11, 2021 at 9:55 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Mon, Oct 11, 2021 at 8:21 AM torikoshia <torikoshia@oss.nttdata.com> wrote:

If we can use debuggers, it's possible to know the memory contexts e.g.
using MemoryContextStats().
So IMHO if it's necessary to know memory contexts without attaching gdb
for other than client backends(probably this means using under
production environment), this enhancement would be pay.

Thanks for providing your thoughts. Knowing memory usage of auxiliary
processes is as important as backends (user session processes) without
attaching debugger in production environments.

There are some open points as mentioned in my first mail in this
thread, I will start working on this patch once we agree on them.

I'm attaching the v1 patch that enables
pg_log_backend_memory_contexts() to log memory contexts of auxiliary
processes. Please review it.

Here's the CF entry - https://commitfest.postgresql.org/35/3385/

After the patch applied the function looks like this

proc = BackendPidGetProc(pid);
if (proc == NULL)
<try aux processes>
<set is_aux_proc>
if (proc == NULL)
<error>
if (!is_aux_proc)
<set local backend id>
SendProcSignal(.., the backend id);

is_aux_proc lookslike making the code complex. I think we can remove
it.

+	/* Only regular backends will have valid backend id, auxiliary processes don't. */
+	if (!is_aux_proc)
+		backendId = proc->backendId;

I think the reason we need to do this is not that aux processes have
the invalid backend id (=InvalidBackendId) but that "some" auxiliary
processes may have a broken proc->backendId in regard to
SendProcSignal (we know that's the startup for now.).

+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('autovacuum launcher'+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('logical replication launcher'));
...

Maybe we can reduce (a quite bit of) run time of the test by
loopingover the processes but since the test only checks if the
function doesn't fail to send a signal, I'm not sure we need to
perform the test for all of the processes here. On the other hand,
the test is missing the most significant target of the startup
process.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

#6Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Kyotaro Horiguchi (#5)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Mon, Nov 1, 2021 at 6:42 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:

At Fri, 29 Oct 2021 22:25:04 +0530, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote in

On Mon, Oct 11, 2021 at 9:55 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Mon, Oct 11, 2021 at 8:21 AM torikoshia <torikoshia@oss.nttdata.com> wrote:

If we can use debuggers, it's possible to know the memory contexts e.g.
using MemoryContextStats().
So IMHO if it's necessary to know memory contexts without attaching gdb
for other than client backends(probably this means using under
production environment), this enhancement would be pay.

Thanks for providing your thoughts. Knowing memory usage of auxiliary
processes is as important as backends (user session processes) without
attaching debugger in production environments.

There are some open points as mentioned in my first mail in this
thread, I will start working on this patch once we agree on them.

I'm attaching the v1 patch that enables
pg_log_backend_memory_contexts() to log memory contexts of auxiliary
processes. Please review it.

Here's the CF entry - https://commitfest.postgresql.org/35/3385/

After the patch applied the function looks like this

proc = BackendPidGetProc(pid);
if (proc == NULL)
<try aux processes>
<set is_aux_proc>
if (proc == NULL)
<error>
if (!is_aux_proc)
<set local backend id>
SendProcSignal(.., the backend id);

is_aux_proc lookslike making the code complex. I think we can remove
it.

+       /* Only regular backends will have valid backend id, auxiliary processes don't. */
+       if (!is_aux_proc)
+               backendId = proc->backendId;

I think the reason we need to do this is not that aux processes have
the invalid backend id (=InvalidBackendId) but that "some" auxiliary
processes may have a broken proc->backendId in regard to
SendProcSignal (we know that's the startup for now.).

I wanted to not have any problems signalling the startup process with
the current code. Yes, the startup process is the only auxiliary
process that has a valid backind id and we have other threads fixing
it. Let's keep the way it is in the v1 patch. Based on whichever patch
gets in we can modify the code.

+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('autovacuum launcher'+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('logical replication launcher'));
...

Maybe we can reduce (a quite bit of) run time of the test by
loopingover the processes but since the test only checks if the
function doesn't fail to send a signal, I'm not sure we need to
perform the test for all of the processes here.

Okay, let me choose the checkpointer for this test, I will remove other tests.

On the other hand,
the test is missing the most significant target of the startup
process.

If we were to have tests for the startup process, then it needs to be
in TAP tests as we have to start a hot standby where the startup
process will be in continuous mode. Is there any other way that we can
add the test case in a .sql file? Do we need to get into this much
complexity for the test case?

Regards,
Bharath Rupireddy.

#7Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Bharath Rupireddy (#6)
1 attachment(s)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Thu, Nov 4, 2021 at 9:35 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

I think the reason we need to do this is not that aux processes have
the invalid backend id (=InvalidBackendId) but that "some" auxiliary
processes may have a broken proc->backendId in regard to
SendProcSignal (we know that's the startup for now.).

I wanted to not have any problems signalling the startup process with
the current code. Yes, the startup process is the only auxiliary
process that has a valid backind id and we have other threads fixing
it. Let's keep the way it is in the v1 patch. Based on whichever patch
gets in we can modify the code.

I added a note there (with XXX) describing the fact that we explicitly
need to send invalid backend id to SendProcSignal.

+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('autovacuum launcher'+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('logical replication launcher'));
...

Maybe we can reduce (a quite bit of) run time of the test by
loopingover the processes but since the test only checks if the
function doesn't fail to send a signal, I'm not sure we need to
perform the test for all of the processes here.

Okay, let me choose the checkpointer for this test, I will remove other tests.

I retained the test case just for the checkpointer.

On the other hand,
the test is missing the most significant target of the startup
process.

If we were to have tests for the startup process, then it needs to be
in TAP tests as we have to start a hot standby where the startup
process will be in continuous mode. Is there any other way that we can
add the test case in a .sql file? Do we need to get into this much
complexity for the test case?

I've not added a TAP test case for the startup process, I see it as
unnecessary. I've tested the startup process case manually here which
just works.

PSA v2 patch and review it.

Regards,
Bharath Rupireddy.

Attachments:

v2-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchapplication/octet-stream; name=v2-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchDownload
From 052819028e3d6b1608ed2deee673f56e27de49cc Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 5 Nov 2021 05:40:10 +0000
Subject: [PATCH v2] enhance pg_log_backend_memory_contexts to log memory
 contexts of aux procs

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of
these processes. Inside the code, we could use the AuxiliaryPidGetProc()
to get the PGPROC of these processes. Note that, neither
AuxiliaryPidGetProc() nor BackendPidGetProc() can return PGPROC(as
they don't have PGPROC entries at all) entries for the syslogger,
stats collector processes.
---
 doc/src/sgml/func.sgml                       | 14 +++++++---
 src/backend/postmaster/checkpointer.c        |  4 +++
 src/backend/postmaster/interrupt.c           |  5 ++++
 src/backend/postmaster/pgarch.c              |  5 ++++
 src/backend/postmaster/startup.c             |  5 ++++
 src/backend/postmaster/walwriter.c           |  4 +++
 src/backend/utils/adt/mcxtfuncs.c            | 29 ++++++++++++++------
 src/test/regress/expected/misc_functions.out | 11 ++++++++
 src/test/regress/sql/misc_functions.sql      |  9 ++++++
 9 files changed, 74 insertions(+), 12 deletions(-)

diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 4b49dff2ff..2449434abf 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25325,10 +25325,16 @@ SELECT collation for ('foo' COLLATE "de_DE");
         <returnvalue>boolean</returnvalue>
        </para>
        <para>
-        Requests to log the memory contexts of the backend with the
-        specified process ID.  These memory contexts will be logged at
-        <literal>LOG</literal> message level. They will appear in
-        the server log based on the log configuration set
+        Requests to log memory contexts of the backend or the
+        <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or
+        the <glossterm linkend="glossary-auxiliary-proc">auxiliary process</glossterm>
+        with the specified process ID. All of the
+        <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>
+        are supported except the <glossterm linkend="glossary-logger">logger</glossterm>
+        and the <glossterm linkend="glossary-stats-collector">statistics collector</glossterm>
+        as they are not connected to shared memory the function can not make requests.
+        These memory contexts will be logged at <literal>LOG</literal> message level.
+        They will appear in the server log based on the log configuration set
         (See <xref linkend="runtime-config-logging"/> for more information),
         but will not be sent to the client regardless of
         <xref linkend="guc-client-min-messages"/>.
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index be7366379d..977f3570aa 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -581,6 +581,10 @@ HandleCheckpointerInterrupts(void)
 		/* Normal exit from the checkpointer is here */
 		proc_exit(0);			/* done */
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/interrupt.c b/src/backend/postmaster/interrupt.c
index dd9136a942..7c1d35a715 100644
--- a/src/backend/postmaster/interrupt.c
+++ b/src/backend/postmaster/interrupt.c
@@ -22,6 +22,7 @@
 #include "storage/latch.h"
 #include "storage/procsignal.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 
 volatile sig_atomic_t ConfigReloadPending = false;
 volatile sig_atomic_t ShutdownRequestPending = false;
@@ -43,6 +44,10 @@ HandleMainLoopInterrupts(void)
 
 	if (ShutdownRequestPending)
 		proc_exit(0);
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 74a7d7c4d0..45f18450ce 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -48,6 +48,7 @@
 #include "storage/procsignal.h"
 #include "storage/shmem.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/ps_status.h"
 
 
@@ -715,4 +716,8 @@ HandlePgArchInterrupts(void)
 		ConfigReloadPending = false;
 		ProcessConfigFile(PGC_SIGHUP);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 47ec737888..2b35a82903 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -32,6 +32,7 @@
 #include "storage/procsignal.h"
 #include "storage/standby.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/timeout.h"
 
 
@@ -200,6 +201,10 @@ HandleStartupProcInterrupts(void)
 	/* Process barrier events */
 	if (ProcSignalBarrierPending)
 		ProcessProcSignalBarrier();
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 626fae8454..19c9118fd8 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -306,4 +306,8 @@ HandleWalWriterInterrupts(void)
 
 		proc_exit(0);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..67598ddbb3 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -176,17 +176,26 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 {
 	int			pid = PG_GETARG_INT32(0);
 	PGPROC	   *proc;
+	bool		is_aux_proc = false;
+	BackendId   backendId = InvalidBackendId;
 
 	proc = BackendPidGetProc(pid);
 
+	/* See if the process with given pid is an auxiliary process. */
+	if (proc == NULL)
+	{
+		proc = AuxiliaryPidGetProc(pid);
+		is_aux_proc = true;
+	}
+
 	/*
-	 * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
-	 * we reach kill(), a process for which we get a valid proc here might
-	 * have terminated on its own.  There's no way to acquire a lock on an
-	 * arbitrary process to prevent that. But since this mechanism is usually
-	 * used to debug a backend running and consuming lots of memory, that it
-	 * might end on its own first and its memory contexts are not logged is
-	 * not a problem.
+	 * BackendPidGetProc or AuxiliaryPidGetProc returns NULL if the pid isn't
+	 * valid; but by the time we reach kill(), a process for which we get a
+	 * valid proc here might have terminated on its own.  There's no way to
+	 * acquire a lock on an arbitrary process to prevent that. But since this
+	 * mechanism is usually used to debug a backend running and consuming lots
+	 * of memory, that it might end on its own first and its memory contexts
+	 * are not logged is not a problem.
 	 */
 	if (proc == NULL)
 	{
@@ -199,7 +208,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 		PG_RETURN_BOOL(false);
 	}
 
-	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
+	/* Only regular backends will have valid backend id, auxiliary processes don't. */
+	if (!is_aux_proc)
+		backendId = proc->backendId;
+
+	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
 	{
 		/* Again, just a warning to allow loops */
 		ereport(WARNING,
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 71d316cad3..ca52779f85 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -147,6 +147,17 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
  t
 (1 row)
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
+DROP FUNCTION memcxt_get_proc_pid(text);
 CREATE ROLE regress_log_memory;
 SELECT has_function_privilege('regress_log_memory',
   'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 8c23874b3f..a9948e59bb 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -41,6 +41,15 @@ SELECT num_nulls();
 
 SELECT pg_log_backend_memory_contexts(pg_backend_pid());
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+
+DROP FUNCTION memcxt_get_proc_pid(text);
+
 CREATE ROLE regress_log_memory;
 
 SELECT has_function_privilege('regress_log_memory',
-- 
2.25.1

#8Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Bharath Rupireddy (#7)
1 attachment(s)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Fri, Nov 5, 2021 at 11:12 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

PSA v2 patch and review it.

I've modified the docs part a bit, please consider v3 for review.

Regards,
Bharath Rupireddy.

Attachments:

v3-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchapplication/octet-stream; name=v3-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchDownload
From 0d22a65edac362005d0c810045ae487cafec95c3 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Mon, 15 Nov 2021 02:13:11 +0000
Subject: [PATCH v3] enhance pg_log_backend_memory_contexts to log memory
 contexts of aux procs

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of
these processes. Inside the code, we could use the AuxiliaryPidGetProc()
to get the PGPROC of these processes. Note that, neither
AuxiliaryPidGetProc() nor BackendPidGetProc() can return PGPROC(as
they don't have PGPROC entries at all) entries for the syslogger,
stats collector processes.
---
 doc/src/sgml/func.sgml                       | 14 +++++++---
 src/backend/postmaster/checkpointer.c        |  4 +++
 src/backend/postmaster/interrupt.c           |  5 ++++
 src/backend/postmaster/pgarch.c              |  5 ++++
 src/backend/postmaster/startup.c             |  5 ++++
 src/backend/postmaster/walwriter.c           |  4 +++
 src/backend/utils/adt/mcxtfuncs.c            | 29 ++++++++++++++------
 src/test/regress/expected/misc_functions.out | 11 ++++++++
 src/test/regress/sql/misc_functions.sql      |  9 ++++++
 9 files changed, 74 insertions(+), 12 deletions(-)

diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 24447c0017..64091030f7 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25335,10 +25335,16 @@ SELECT collation for ('foo' COLLATE "de_DE");
         <returnvalue>boolean</returnvalue>
        </para>
        <para>
-        Requests to log the memory contexts of the backend with the
-        specified process ID.  These memory contexts will be logged at
-        <literal>LOG</literal> message level. They will appear in
-        the server log based on the log configuration set
+        Requests to log memory contexts of the <glossterm linkend="glossary-backend">backend</glossterm>
+        or the <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or
+        the <glossterm linkend="glossary-auxiliary-proc">auxiliary process</glossterm>
+        with the specified process ID. All of the
+        <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>
+        are supported except the <glossterm linkend="glossary-logger">logger</glossterm>
+        and the <glossterm linkend="glossary-stats-collector">statistics collector</glossterm>
+        as they are not connected to shared memory the function can not make requests.
+        These memory contexts will be logged at <literal>LOG</literal> message level.
+        They will appear in the server log based on the log configuration set
         (See <xref linkend="runtime-config-logging"/> for more information),
         but will not be sent to the client regardless of
         <xref linkend="guc-client-min-messages"/>.
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index be7366379d..977f3570aa 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -581,6 +581,10 @@ HandleCheckpointerInterrupts(void)
 		/* Normal exit from the checkpointer is here */
 		proc_exit(0);			/* done */
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/interrupt.c b/src/backend/postmaster/interrupt.c
index dd9136a942..7c1d35a715 100644
--- a/src/backend/postmaster/interrupt.c
+++ b/src/backend/postmaster/interrupt.c
@@ -22,6 +22,7 @@
 #include "storage/latch.h"
 #include "storage/procsignal.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 
 volatile sig_atomic_t ConfigReloadPending = false;
 volatile sig_atomic_t ShutdownRequestPending = false;
@@ -43,6 +44,10 @@ HandleMainLoopInterrupts(void)
 
 	if (ShutdownRequestPending)
 		proc_exit(0);
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 3b33e01d95..eb42682a74 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -50,6 +50,7 @@
 #include "storage/shmem.h"
 #include "storage/spin.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/ps_status.h"
 
 
@@ -856,4 +857,8 @@ HandlePgArchInterrupts(void)
 		ConfigReloadPending = false;
 		ProcessConfigFile(PGC_SIGHUP);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 47ec737888..2b35a82903 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -32,6 +32,7 @@
 #include "storage/procsignal.h"
 #include "storage/standby.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/timeout.h"
 
 
@@ -200,6 +201,10 @@ HandleStartupProcInterrupts(void)
 	/* Process barrier events */
 	if (ProcSignalBarrierPending)
 		ProcessProcSignalBarrier();
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 626fae8454..19c9118fd8 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -306,4 +306,8 @@ HandleWalWriterInterrupts(void)
 
 		proc_exit(0);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..67598ddbb3 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -176,17 +176,26 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 {
 	int			pid = PG_GETARG_INT32(0);
 	PGPROC	   *proc;
+	bool		is_aux_proc = false;
+	BackendId   backendId = InvalidBackendId;
 
 	proc = BackendPidGetProc(pid);
 
+	/* See if the process with given pid is an auxiliary process. */
+	if (proc == NULL)
+	{
+		proc = AuxiliaryPidGetProc(pid);
+		is_aux_proc = true;
+	}
+
 	/*
-	 * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
-	 * we reach kill(), a process for which we get a valid proc here might
-	 * have terminated on its own.  There's no way to acquire a lock on an
-	 * arbitrary process to prevent that. But since this mechanism is usually
-	 * used to debug a backend running and consuming lots of memory, that it
-	 * might end on its own first and its memory contexts are not logged is
-	 * not a problem.
+	 * BackendPidGetProc or AuxiliaryPidGetProc returns NULL if the pid isn't
+	 * valid; but by the time we reach kill(), a process for which we get a
+	 * valid proc here might have terminated on its own.  There's no way to
+	 * acquire a lock on an arbitrary process to prevent that. But since this
+	 * mechanism is usually used to debug a backend running and consuming lots
+	 * of memory, that it might end on its own first and its memory contexts
+	 * are not logged is not a problem.
 	 */
 	if (proc == NULL)
 	{
@@ -199,7 +208,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 		PG_RETURN_BOOL(false);
 	}
 
-	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
+	/* Only regular backends will have valid backend id, auxiliary processes don't. */
+	if (!is_aux_proc)
+		backendId = proc->backendId;
+
+	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
 	{
 		/* Again, just a warning to allow loops */
 		ereport(WARNING,
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 71d316cad3..ca52779f85 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -147,6 +147,17 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
  t
 (1 row)
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
+DROP FUNCTION memcxt_get_proc_pid(text);
 CREATE ROLE regress_log_memory;
 SELECT has_function_privilege('regress_log_memory',
   'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 8c23874b3f..a9948e59bb 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -41,6 +41,15 @@ SELECT num_nulls();
 
 SELECT pg_log_backend_memory_contexts(pg_backend_pid());
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+
+DROP FUNCTION memcxt_get_proc_pid(text);
+
 CREATE ROLE regress_log_memory;
 
 SELECT has_function_privilege('regress_log_memory',
-- 
2.25.1

#9vignesh C
vignesh21@gmail.com
In reply to: Bharath Rupireddy (#8)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Mon, Nov 15, 2021 at 7:47 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Fri, Nov 5, 2021 at 11:12 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

PSA v2 patch and review it.

I've modified the docs part a bit, please consider v3 for review.

Thanks for the update patch, Few comments:
1) Should we change "CHECK_FOR_INTERRUPTS()" to
"CHECK_FOR_INTERRUPTS() or process specific interrupt handlers"
/*
* pg_log_backend_memory_contexts
* Signal a backend process to log its memory contexts.
*
* By default, only superusers are allowed to signal to log the memory
* contexts because allowing any users to issue this request at an unbounded
* rate would cause lots of log messages and which can lead to denial of
* service. Additional roles can be permitted with GRANT.
*
* On receipt of this signal, a backend sets the flag in the signal
* handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
* memory contexts.
*/
Datum
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)

2) Should we mention Postmaster process also along with logger and
statistics collector process
+        <glossterm linkend="glossary-backend">backend</glossterm> or the
+        <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or the
+        <glossterm linkend="glossary-auxiliary-proc">auxiliary
process</glossterm>
+        with the specified process ID. All of the
+        <glossterm linkend="glossary-auxiliary-proc">auxiliary
processes</glossterm>
+        are supported except the <glossterm
linkend="glossary-logger">logger</glossterm>
+        and the <glossterm
linkend="glossary-stats-collector">statistics collector</glossterm>
+        as they are not connected to shared memory the function can
not make requests.
+        The backtrace will be logged at <literal>LOG</literal> message level.
+        They will appear in the server log based on the log configuration set
+        (See <xref linkend="runtime-config-logging"/> for more information),
+        but will not be sent to the client regardless of

Regards,
Vignesh

#10Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: vignesh C (#9)
1 attachment(s)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Mon, Nov 15, 2021 at 10:04 PM vignesh C <vignesh21@gmail.com> wrote:

On Mon, Nov 15, 2021 at 7:47 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Fri, Nov 5, 2021 at 11:12 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

PSA v2 patch and review it.

I've modified the docs part a bit, please consider v3 for review.

Thanks for the update patch, Few comments:
1) Should we change "CHECK_FOR_INTERRUPTS()" to
"CHECK_FOR_INTERRUPTS() or process specific interrupt handlers"

Done.

2) Should we mention Postmaster process also along with logger and
statistics collector process
+        <glossterm linkend="glossary-backend">backend</glossterm> or the
+        <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or the
+        <glossterm linkend="glossary-auxiliary-proc">auxiliary
process</glossterm>
+        with the specified process ID. All of the
+        <glossterm linkend="glossary-auxiliary-proc">auxiliary
processes</glossterm>
+        are supported except the <glossterm
linkend="glossary-logger">logger</glossterm>
+        and the <glossterm
linkend="glossary-stats-collector">statistics collector</glossterm>
+        as they are not connected to shared memory the function can
not make requests.
+        The backtrace will be logged at <literal>LOG</literal> message level.
+        They will appear in the server log based on the log configuration set
+        (See <xref linkend="runtime-config-logging"/> for more information),
+        but will not be sent to the client regardless of

Done.

Attaching v4 patch, please review it further.

Regards,
Bharath Rupireddy.

Attachments:

v4-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchapplication/octet-stream; name=v4-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchDownload
From 295335122c79706d520a543f821f918348064ced Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Mon, 15 Nov 2021 16:55:22 +0000
Subject: [PATCH v4] enhance pg_log_backend_memory_contexts to log memory
 contexts of aux procs

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of
these processes. Inside the code, we could use the AuxiliaryPidGetProc()
to get the PGPROC of these processes. Note that, neither
AuxiliaryPidGetProc() nor BackendPidGetProc() can return PGPROC(as
they don't have PGPROC entries at all) entries for the syslogger,
stats collector processes.
---
 doc/src/sgml/func.sgml                       | 18 ++++++----
 src/backend/postmaster/checkpointer.c        |  4 +++
 src/backend/postmaster/interrupt.c           |  5 +++
 src/backend/postmaster/pgarch.c              |  5 +++
 src/backend/postmaster/startup.c             |  5 +++
 src/backend/postmaster/walwriter.c           |  4 +++
 src/backend/utils/adt/mcxtfuncs.c            | 35 ++++++++++++++------
 src/test/regress/expected/misc_functions.out | 11 ++++++
 src/test/regress/sql/misc_functions.sql      |  9 +++++
 9 files changed, 79 insertions(+), 17 deletions(-)

diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 24447c0017..78ae0d2964 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25335,12 +25335,18 @@ SELECT collation for ('foo' COLLATE "de_DE");
         <returnvalue>boolean</returnvalue>
        </para>
        <para>
-        Requests to log the memory contexts of the backend with the
-        specified process ID.  These memory contexts will be logged at
-        <literal>LOG</literal> message level. They will appear in
-        the server log based on the log configuration set
-        (See <xref linkend="runtime-config-logging"/> for more information),
-        but will not be sent to the client regardless of
+        Requests to log memory contexts of the <glossterm linkend="glossary-backend">backend</glossterm>
+        or the <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or
+        the <glossterm linkend="glossary-auxiliary-proc">auxiliary process</glossterm>
+        with the specified process ID. This function cannot request
+        <glossterm linkend="glossary-postmaster">postmaster process</glossterm> or
+        <glossterm linkend="glossary-logger">logger</glossterm> or 
+        <glossterm linkend="glossary-stats-collector">statistics collector</glossterm>
+        (all other <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>
+        it can) for memory contexts. These memory contexts will be logged at
+        <literal>LOG</literal>message level. They will appear in the server log
+        based on the log configuration set (See <xref linkend="runtime-config-logging"/>
+        for more information), but will not be sent to the client regardless of
         <xref linkend="guc-client-min-messages"/>.
        </para></entry>
       </row>
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index be7366379d..977f3570aa 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -581,6 +581,10 @@ HandleCheckpointerInterrupts(void)
 		/* Normal exit from the checkpointer is here */
 		proc_exit(0);			/* done */
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/interrupt.c b/src/backend/postmaster/interrupt.c
index dd9136a942..7c1d35a715 100644
--- a/src/backend/postmaster/interrupt.c
+++ b/src/backend/postmaster/interrupt.c
@@ -22,6 +22,7 @@
 #include "storage/latch.h"
 #include "storage/procsignal.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 
 volatile sig_atomic_t ConfigReloadPending = false;
 volatile sig_atomic_t ShutdownRequestPending = false;
@@ -43,6 +44,10 @@ HandleMainLoopInterrupts(void)
 
 	if (ShutdownRequestPending)
 		proc_exit(0);
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 3b33e01d95..eb42682a74 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -50,6 +50,7 @@
 #include "storage/shmem.h"
 #include "storage/spin.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/ps_status.h"
 
 
@@ -856,4 +857,8 @@ HandlePgArchInterrupts(void)
 		ConfigReloadPending = false;
 		ProcessConfigFile(PGC_SIGHUP);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 47ec737888..2b35a82903 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -32,6 +32,7 @@
 #include "storage/procsignal.h"
 #include "storage/standby.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/timeout.h"
 
 
@@ -200,6 +201,10 @@ HandleStartupProcInterrupts(void)
 	/* Process barrier events */
 	if (ProcSignalBarrierPending)
 		ProcessProcSignalBarrier();
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 626fae8454..19c9118fd8 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -306,4 +306,8 @@ HandleWalWriterInterrupts(void)
 
 		proc_exit(0);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..377b366927 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -167,26 +167,35 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
  * rate would cause lots of log messages and which can lead to denial of
  * service. Additional roles can be permitted with GRANT.
  *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
+ * On receipt of this signal, a backend sets the flag in the signal handler,
+ * which causes the next CHECK_FOR_INTERRUPTS() or process specific interrupt
+ * handlers to log the memory contexts.
  */
 Datum
 pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 {
 	int			pid = PG_GETARG_INT32(0);
 	PGPROC	   *proc;
+	bool		is_aux_proc = false;
+	BackendId   backendId = InvalidBackendId;
 
 	proc = BackendPidGetProc(pid);
 
+	/* See if the process with given pid is an auxiliary process. */
+	if (proc == NULL)
+	{
+		proc = AuxiliaryPidGetProc(pid);
+		is_aux_proc = true;
+	}
+
 	/*
-	 * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
-	 * we reach kill(), a process for which we get a valid proc here might
-	 * have terminated on its own.  There's no way to acquire a lock on an
-	 * arbitrary process to prevent that. But since this mechanism is usually
-	 * used to debug a backend running and consuming lots of memory, that it
-	 * might end on its own first and its memory contexts are not logged is
-	 * not a problem.
+	 * BackendPidGetProc or AuxiliaryPidGetProc returns NULL if the pid isn't
+	 * valid; but by the time we reach kill(), a process for which we get a
+	 * valid proc here might have terminated on its own.  There's no way to
+	 * acquire a lock on an arbitrary process to prevent that. But since this
+	 * mechanism is usually used to debug a backend running and consuming lots
+	 * of memory, that it might end on its own first and its memory contexts
+	 * are not logged is not a problem.
 	 */
 	if (proc == NULL)
 	{
@@ -199,7 +208,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 		PG_RETURN_BOOL(false);
 	}
 
-	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
+	/* Only regular backends will have valid backend id, auxiliary processes don't. */
+	if (!is_aux_proc)
+		backendId = proc->backendId;
+
+	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
 	{
 		/* Again, just a warning to allow loops */
 		ereport(WARNING,
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 71d316cad3..ca52779f85 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -147,6 +147,17 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
  t
 (1 row)
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
+DROP FUNCTION memcxt_get_proc_pid(text);
 CREATE ROLE regress_log_memory;
 SELECT has_function_privilege('regress_log_memory',
   'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 8c23874b3f..a9948e59bb 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -41,6 +41,15 @@ SELECT num_nulls();
 
 SELECT pg_log_backend_memory_contexts(pg_backend_pid());
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+
+DROP FUNCTION memcxt_get_proc_pid(text);
+
 CREATE ROLE regress_log_memory;
 
 SELECT has_function_privilege('regress_log_memory',
-- 
2.25.1

#11vignesh C
vignesh21@gmail.com
In reply to: Bharath Rupireddy (#10)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Mon, Nov 15, 2021 at 10:27 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Mon, Nov 15, 2021 at 10:04 PM vignesh C <vignesh21@gmail.com> wrote:

On Mon, Nov 15, 2021 at 7:47 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Fri, Nov 5, 2021 at 11:12 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

PSA v2 patch and review it.

I've modified the docs part a bit, please consider v3 for review.

Thanks for the update patch, Few comments:
1) Should we change "CHECK_FOR_INTERRUPTS()" to
"CHECK_FOR_INTERRUPTS() or process specific interrupt handlers"

Done.

2) Should we mention Postmaster process also along with logger and
statistics collector process
+        <glossterm linkend="glossary-backend">backend</glossterm> or the
+        <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or the
+        <glossterm linkend="glossary-auxiliary-proc">auxiliary
process</glossterm>
+        with the specified process ID. All of the
+        <glossterm linkend="glossary-auxiliary-proc">auxiliary
processes</glossterm>
+        are supported except the <glossterm
linkend="glossary-logger">logger</glossterm>
+        and the <glossterm
linkend="glossary-stats-collector">statistics collector</glossterm>
+        as they are not connected to shared memory the function can
not make requests.
+        The backtrace will be logged at <literal>LOG</literal> message level.
+        They will appear in the server log based on the log configuration set
+        (See <xref linkend="runtime-config-logging"/> for more information),
+        but will not be sent to the client regardless of

Done.

Attaching v4 patch, please review it further.

One small comment:
1) There should be a space in between "<literal>LOG</literal>message level"
+        it can) for memory contexts. These memory contexts will be logged at
+        <literal>LOG</literal>message level. They will appear in the server log
+        based on the log configuration set (See <xref
linkend="runtime-config-logging"/>
+        for more information), but will not be sent to the client regardless of

The rest of the patch looks good to me.

Regards,
Vignesh

#12Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: vignesh C (#11)
1 attachment(s)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Sun, Nov 28, 2021 at 12:22 PM vignesh C <vignesh21@gmail.com> wrote:

Attaching v4 patch, please review it further.

One small comment:
1) There should be a space in between "<literal>LOG</literal>message level"
+        it can) for memory contexts. These memory contexts will be logged at
+        <literal>LOG</literal>message level. They will appear in the server log
+        based on the log configuration set (See <xref
linkend="runtime-config-logging"/>
+        for more information), but will not be sent to the client regardless of

Done.

The rest of the patch looks good to me.

Thanks for the review. Here's the v5 patch.

Regards,
Bharath Rupireddy.

Attachments:

v5-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchapplication/octet-stream; name=v5-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchDownload
From bd533ed3adf8e10aadb8055a796ac43dbe70f613 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Sun, 28 Nov 2021 06:54:36 +0000
Subject: [PATCH v5] enhance pg_log_backend_memory_contexts to log memory
 contexts of aux procs

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of
these processes. Inside the code, we could use the AuxiliaryPidGetProc()
to get the PGPROC of these processes. Note that, neither
AuxiliaryPidGetProc() nor BackendPidGetProc() can return PGPROC(as
they don't have PGPROC entries at all) entries for the syslogger,
stats collector processes.
---
 doc/src/sgml/func.sgml                       | 18 ++++++----
 src/backend/postmaster/checkpointer.c        |  4 +++
 src/backend/postmaster/interrupt.c           |  5 +++
 src/backend/postmaster/pgarch.c              |  5 +++
 src/backend/postmaster/startup.c             |  5 +++
 src/backend/postmaster/walwriter.c           |  4 +++
 src/backend/utils/adt/mcxtfuncs.c            | 35 ++++++++++++++------
 src/test/regress/expected/misc_functions.out | 11 ++++++
 src/test/regress/sql/misc_functions.sql      |  9 +++++
 9 files changed, 79 insertions(+), 17 deletions(-)

diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 0a725a6711..0dce33712e 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25348,12 +25348,18 @@ SELECT collation for ('foo' COLLATE "de_DE");
         <returnvalue>boolean</returnvalue>
        </para>
        <para>
-        Requests to log the memory contexts of the backend with the
-        specified process ID.  These memory contexts will be logged at
-        <literal>LOG</literal> message level. They will appear in
-        the server log based on the log configuration set
-        (See <xref linkend="runtime-config-logging"/> for more information),
-        but will not be sent to the client regardless of
+        Requests to log memory contexts of the <glossterm linkend="glossary-backend">backend</glossterm>
+        or the <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or
+        the <glossterm linkend="glossary-auxiliary-proc">auxiliary process</glossterm>
+        with the specified process ID. This function cannot request
+        <glossterm linkend="glossary-postmaster">postmaster process</glossterm> or
+        <glossterm linkend="glossary-logger">logger</glossterm> or 
+        <glossterm linkend="glossary-stats-collector">statistics collector</glossterm>
+        (all other <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>
+        it can) for memory contexts. These memory contexts will be logged at
+        <literal>LOG</literal> message level. They will appear in the server log
+        based on the log configuration set (See <xref linkend="runtime-config-logging"/>
+        for more information), but will not be sent to the client regardless of
         <xref linkend="guc-client-min-messages"/>.
        </para></entry>
       </row>
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index be7366379d..977f3570aa 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -581,6 +581,10 @@ HandleCheckpointerInterrupts(void)
 		/* Normal exit from the checkpointer is here */
 		proc_exit(0);			/* done */
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/interrupt.c b/src/backend/postmaster/interrupt.c
index dd9136a942..7c1d35a715 100644
--- a/src/backend/postmaster/interrupt.c
+++ b/src/backend/postmaster/interrupt.c
@@ -22,6 +22,7 @@
 #include "storage/latch.h"
 #include "storage/procsignal.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 
 volatile sig_atomic_t ConfigReloadPending = false;
 volatile sig_atomic_t ShutdownRequestPending = false;
@@ -43,6 +44,10 @@ HandleMainLoopInterrupts(void)
 
 	if (ShutdownRequestPending)
 		proc_exit(0);
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 434939be9b..464e21d20d 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -50,6 +50,7 @@
 #include "storage/shmem.h"
 #include "storage/spin.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/ps_status.h"
 
 
@@ -859,4 +860,8 @@ HandlePgArchInterrupts(void)
 		ConfigReloadPending = false;
 		ProcessConfigFile(PGC_SIGHUP);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 47ec737888..2b35a82903 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -32,6 +32,7 @@
 #include "storage/procsignal.h"
 #include "storage/standby.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/timeout.h"
 
 
@@ -200,6 +201,10 @@ HandleStartupProcInterrupts(void)
 	/* Process barrier events */
 	if (ProcSignalBarrierPending)
 		ProcessProcSignalBarrier();
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 626fae8454..19c9118fd8 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -306,4 +306,8 @@ HandleWalWriterInterrupts(void)
 
 		proc_exit(0);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..377b366927 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -167,26 +167,35 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
  * rate would cause lots of log messages and which can lead to denial of
  * service. Additional roles can be permitted with GRANT.
  *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
+ * On receipt of this signal, a backend sets the flag in the signal handler,
+ * which causes the next CHECK_FOR_INTERRUPTS() or process specific interrupt
+ * handlers to log the memory contexts.
  */
 Datum
 pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 {
 	int			pid = PG_GETARG_INT32(0);
 	PGPROC	   *proc;
+	bool		is_aux_proc = false;
+	BackendId   backendId = InvalidBackendId;
 
 	proc = BackendPidGetProc(pid);
 
+	/* See if the process with given pid is an auxiliary process. */
+	if (proc == NULL)
+	{
+		proc = AuxiliaryPidGetProc(pid);
+		is_aux_proc = true;
+	}
+
 	/*
-	 * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
-	 * we reach kill(), a process for which we get a valid proc here might
-	 * have terminated on its own.  There's no way to acquire a lock on an
-	 * arbitrary process to prevent that. But since this mechanism is usually
-	 * used to debug a backend running and consuming lots of memory, that it
-	 * might end on its own first and its memory contexts are not logged is
-	 * not a problem.
+	 * BackendPidGetProc or AuxiliaryPidGetProc returns NULL if the pid isn't
+	 * valid; but by the time we reach kill(), a process for which we get a
+	 * valid proc here might have terminated on its own.  There's no way to
+	 * acquire a lock on an arbitrary process to prevent that. But since this
+	 * mechanism is usually used to debug a backend running and consuming lots
+	 * of memory, that it might end on its own first and its memory contexts
+	 * are not logged is not a problem.
 	 */
 	if (proc == NULL)
 	{
@@ -199,7 +208,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 		PG_RETURN_BOOL(false);
 	}
 
-	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
+	/* Only regular backends will have valid backend id, auxiliary processes don't. */
+	if (!is_aux_proc)
+		backendId = proc->backendId;
+
+	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
 	{
 		/* Again, just a warning to allow loops */
 		ereport(WARNING,
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 1013d17f87..6377e432c0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -147,6 +147,17 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
  t
 (1 row)
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
+DROP FUNCTION memcxt_get_proc_pid(text);
 CREATE ROLE regress_log_memory;
 SELECT has_function_privilege('regress_log_memory',
   'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 7ab9b2a150..20b5e692bd 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -41,6 +41,15 @@ SELECT num_nulls();
 
 SELECT pg_log_backend_memory_contexts(pg_backend_pid());
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+
+DROP FUNCTION memcxt_get_proc_pid(text);
+
 CREATE ROLE regress_log_memory;
 
 SELECT has_function_privilege('regress_log_memory',
-- 
2.25.1

#13vignesh C
vignesh21@gmail.com
In reply to: Bharath Rupireddy (#12)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Sun, Nov 28, 2021 at 12:25 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Sun, Nov 28, 2021 at 12:22 PM vignesh C <vignesh21@gmail.com> wrote:

Attaching v4 patch, please review it further.

One small comment:
1) There should be a space in between "<literal>LOG</literal>message level"
+        it can) for memory contexts. These memory contexts will be logged at
+        <literal>LOG</literal>message level. They will appear in the server log
+        based on the log configuration set (See <xref
linkend="runtime-config-logging"/>
+        for more information), but will not be sent to the client regardless of

Done.

The rest of the patch looks good to me.

Thanks for the review. Here's the v5 patch.

Thanks for the updated patch, one comment:
1)  The function can be indented similar to other functions in the same file:
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
Something like:
+CREATE FUNCTION memcxt_get_proc_pid(text)
+  RETURNS int
+  LANGUAGE SQL
+  AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';

Regards,
Vignesh

#14Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: vignesh C (#13)
1 attachment(s)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Sun, Nov 28, 2021 at 5:21 PM vignesh C <vignesh21@gmail.com> wrote:

Thanks for the updated patch, one comment:
1)  The function can be indented similar to other functions in the same file:
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
Something like:
+CREATE FUNCTION memcxt_get_proc_pid(text)
+  RETURNS int
+  LANGUAGE SQL
+  AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';

Done. PSA v6 patch.

Regards,
Bharath Rupireddy.

Attachments:

v6-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchapplication/octet-stream; name=v6-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchDownload
From 0510781eece8d3fc3fd2d2dcf1f5ae752b9d9e21 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Sun, 28 Nov 2021 13:43:35 +0000
Subject: [PATCH v6] enhance pg_log_backend_memory_contexts to log memory
 contexts of aux procs

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of
these processes. Inside the code, we could use the AuxiliaryPidGetProc()
to get the PGPROC of these processes. Note that, neither
AuxiliaryPidGetProc() nor BackendPidGetProc() can return PGPROC(as
they don't have PGPROC entries at all) entries for the syslogger,
stats collector processes.
---
 doc/src/sgml/func.sgml                       | 18 ++++++----
 src/backend/postmaster/checkpointer.c        |  4 +++
 src/backend/postmaster/interrupt.c           |  5 +++
 src/backend/postmaster/pgarch.c              |  5 +++
 src/backend/postmaster/startup.c             |  5 +++
 src/backend/postmaster/walwriter.c           |  4 +++
 src/backend/utils/adt/mcxtfuncs.c            | 35 ++++++++++++++------
 src/test/regress/expected/misc_functions.out | 11 ++++++
 src/test/regress/sql/misc_functions.sql      |  9 +++++
 9 files changed, 79 insertions(+), 17 deletions(-)

diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 0a725a6711..0dce33712e 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25348,12 +25348,18 @@ SELECT collation for ('foo' COLLATE "de_DE");
         <returnvalue>boolean</returnvalue>
        </para>
        <para>
-        Requests to log the memory contexts of the backend with the
-        specified process ID.  These memory contexts will be logged at
-        <literal>LOG</literal> message level. They will appear in
-        the server log based on the log configuration set
-        (See <xref linkend="runtime-config-logging"/> for more information),
-        but will not be sent to the client regardless of
+        Requests to log memory contexts of the <glossterm linkend="glossary-backend">backend</glossterm>
+        or the <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or
+        the <glossterm linkend="glossary-auxiliary-proc">auxiliary process</glossterm>
+        with the specified process ID. This function cannot request
+        <glossterm linkend="glossary-postmaster">postmaster process</glossterm> or
+        <glossterm linkend="glossary-logger">logger</glossterm> or 
+        <glossterm linkend="glossary-stats-collector">statistics collector</glossterm>
+        (all other <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>
+        it can) for memory contexts. These memory contexts will be logged at
+        <literal>LOG</literal> message level. They will appear in the server log
+        based on the log configuration set (See <xref linkend="runtime-config-logging"/>
+        for more information), but will not be sent to the client regardless of
         <xref linkend="guc-client-min-messages"/>.
        </para></entry>
       </row>
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index be7366379d..977f3570aa 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -581,6 +581,10 @@ HandleCheckpointerInterrupts(void)
 		/* Normal exit from the checkpointer is here */
 		proc_exit(0);			/* done */
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/interrupt.c b/src/backend/postmaster/interrupt.c
index dd9136a942..7c1d35a715 100644
--- a/src/backend/postmaster/interrupt.c
+++ b/src/backend/postmaster/interrupt.c
@@ -22,6 +22,7 @@
 #include "storage/latch.h"
 #include "storage/procsignal.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 
 volatile sig_atomic_t ConfigReloadPending = false;
 volatile sig_atomic_t ShutdownRequestPending = false;
@@ -43,6 +44,10 @@ HandleMainLoopInterrupts(void)
 
 	if (ShutdownRequestPending)
 		proc_exit(0);
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 434939be9b..464e21d20d 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -50,6 +50,7 @@
 #include "storage/shmem.h"
 #include "storage/spin.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/ps_status.h"
 
 
@@ -859,4 +860,8 @@ HandlePgArchInterrupts(void)
 		ConfigReloadPending = false;
 		ProcessConfigFile(PGC_SIGHUP);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 47ec737888..2b35a82903 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -32,6 +32,7 @@
 #include "storage/procsignal.h"
 #include "storage/standby.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/timeout.h"
 
 
@@ -200,6 +201,10 @@ HandleStartupProcInterrupts(void)
 	/* Process barrier events */
 	if (ProcSignalBarrierPending)
 		ProcessProcSignalBarrier();
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 626fae8454..19c9118fd8 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -306,4 +306,8 @@ HandleWalWriterInterrupts(void)
 
 		proc_exit(0);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..377b366927 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -167,26 +167,35 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
  * rate would cause lots of log messages and which can lead to denial of
  * service. Additional roles can be permitted with GRANT.
  *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
+ * On receipt of this signal, a backend sets the flag in the signal handler,
+ * which causes the next CHECK_FOR_INTERRUPTS() or process specific interrupt
+ * handlers to log the memory contexts.
  */
 Datum
 pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 {
 	int			pid = PG_GETARG_INT32(0);
 	PGPROC	   *proc;
+	bool		is_aux_proc = false;
+	BackendId   backendId = InvalidBackendId;
 
 	proc = BackendPidGetProc(pid);
 
+	/* See if the process with given pid is an auxiliary process. */
+	if (proc == NULL)
+	{
+		proc = AuxiliaryPidGetProc(pid);
+		is_aux_proc = true;
+	}
+
 	/*
-	 * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
-	 * we reach kill(), a process for which we get a valid proc here might
-	 * have terminated on its own.  There's no way to acquire a lock on an
-	 * arbitrary process to prevent that. But since this mechanism is usually
-	 * used to debug a backend running and consuming lots of memory, that it
-	 * might end on its own first and its memory contexts are not logged is
-	 * not a problem.
+	 * BackendPidGetProc or AuxiliaryPidGetProc returns NULL if the pid isn't
+	 * valid; but by the time we reach kill(), a process for which we get a
+	 * valid proc here might have terminated on its own.  There's no way to
+	 * acquire a lock on an arbitrary process to prevent that. But since this
+	 * mechanism is usually used to debug a backend running and consuming lots
+	 * of memory, that it might end on its own first and its memory contexts
+	 * are not logged is not a problem.
 	 */
 	if (proc == NULL)
 	{
@@ -199,7 +208,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 		PG_RETURN_BOOL(false);
 	}
 
-	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
+	/* Only regular backends will have valid backend id, auxiliary processes don't. */
+	if (!is_aux_proc)
+		backendId = proc->backendId;
+
+	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
 	{
 		/* Again, just a warning to allow loops */
 		ereport(WARNING,
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 1013d17f87..53e691a6ca 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -147,6 +147,17 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
  t
 (1 row)
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+  RETURNS int
+  LANGUAGE SQL
+  AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
+DROP FUNCTION memcxt_get_proc_pid(text);
 CREATE ROLE regress_log_memory;
 SELECT has_function_privilege('regress_log_memory',
   'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 7ab9b2a150..b0db4e09d4 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -41,6 +41,15 @@ SELECT num_nulls();
 
 SELECT pg_log_backend_memory_contexts(pg_backend_pid());
 
+CREATE FUNCTION memcxt_get_proc_pid(text)
+  RETURNS int
+  LANGUAGE SQL
+  AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+
+DROP FUNCTION memcxt_get_proc_pid(text);
+
 CREATE ROLE regress_log_memory;
 
 SELECT has_function_privilege('regress_log_memory',
-- 
2.25.1

#15vignesh C
vignesh21@gmail.com
In reply to: Bharath Rupireddy (#14)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Sun, Nov 28, 2021 at 7:15 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

On Sun, Nov 28, 2021 at 5:21 PM vignesh C <vignesh21@gmail.com> wrote:

Thanks for the updated patch, one comment:
1)  The function can be indented similar to other functions in the same file:
+CREATE FUNCTION memcxt_get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
Something like:
+CREATE FUNCTION memcxt_get_proc_pid(text)
+  RETURNS int
+  LANGUAGE SQL
+  AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';

Done. PSA v6 patch.

Thanks for the updated patch. The patch applies neatly, make
check-world passes and the documentation looks good. I did not find
any issues with the v6 patch, I'm marking the patch as Ready for
Committer.

Regards,
Vignesh

#16Fujii Masao
masao.fujii@oss.nttdata.com
In reply to: vignesh C (#15)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On 2021/11/29 11:44, vignesh C wrote:

Thanks for the updated patch. The patch applies neatly, make
check-world passes and the documentation looks good. I did not find
any issues with the v6 patch, I'm marking the patch as Ready for
Committer.

I started reading the patch.

+CREATE FUNCTION memcxt_get_proc_pid(text)
+  RETURNS int
+  LANGUAGE SQL
+  AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+
+DROP FUNCTION memcxt_get_proc_pid(text);

Why is memcxt_get_proc_pid() still necessary? ISTM that we can just replace the above with the following query, instead.

SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity WHERE backend_type = 'checkpointer'

-        Requests to log the memory contexts of the backend with the
-        specified process ID.  These memory contexts will be logged at
-        <literal>LOG</literal> message level. They will appear in
-        the server log based on the log configuration set
-        (See <xref linkend="runtime-config-logging"/> for more information),
-        but will not be sent to the client regardless of
+        Requests to log memory contexts of the <glossterm linkend="glossary-backend">backend</glossterm>
+        or the <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or
+        the <glossterm linkend="glossary-auxiliary-proc">auxiliary process</glossterm>
+        with the specified process ID. This function cannot request
+        <glossterm linkend="glossary-postmaster">postmaster process</glossterm> or
+        <glossterm linkend="glossary-logger">logger</glossterm> or
+        <glossterm linkend="glossary-stats-collector">statistics collector</glossterm>
+        (all other <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>

ISTM that you're trying to list all possible processes that pg_log_backend_memory_contexts() can handle. But why didn't you list autovacuum worker (while other special backend, WAL sender, is picked up) and background worker like logical replication launcher? Because the term "backend" implicitly includes those processes? If so, why did you pick up WAL sender separately?

I'm tempted to replace these descriptions as follows. Because the following looks simpler and easier to read and understand, to me.

----------------------
Requests to log the memory contexts of the process with the specified process ID. Possible processes that this function can send the request to are: backend, WAL sender, autovacuum worker, auxiliary processes except logger and stats collector, and background workers.
----------------------

or

----------------------
Requests to log the memory contexts of the backend with the specified process ID. This function can send the request to also auxiliary processes except logger and stats collector.
----------------------

+	/* See if the process with given pid is an auxiliary process. */
+	if (proc == NULL)
+	{
+		proc = AuxiliaryPidGetProc(pid);
+		is_aux_proc = true;
+	}

As Horiguchi-san told upthread, IMO it's simpler not to use is_aux_proc flag. For example, you can replace this code with

------------------------
proc = BackendPidGetProc(pid);

if (proc != NULL)
backendId = proc->backendId;
else
proc = AuxiliaryPidGetProc(pid);
------------------------

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

#17Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Fujii Masao (#16)
1 attachment(s)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On Fri, Jan 7, 2022 at 9:19 PM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:

On 2021/11/29 11:44, vignesh C wrote:

Thanks for the updated patch. The patch applies neatly, make
check-world passes and the documentation looks good. I did not find
any issues with the v6 patch, I'm marking the patch as Ready for
Committer.

I started reading the patch.

Thanks.

+CREATE FUNCTION memcxt_get_proc_pid(text)
+  RETURNS int
+  LANGUAGE SQL
+  AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+
+SELECT pg_log_backend_memory_contexts(memcxt_get_proc_pid('checkpointer'));
+
+DROP FUNCTION memcxt_get_proc_pid(text);

Why is memcxt_get_proc_pid() still necessary? ISTM that we can just replace the above with the following query, instead.

SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity WHERE backend_type = 'checkpointer'

Changed.

I'm tempted to replace these descriptions as follows. Because the following looks simpler and easier to read and understand, to me.
----------------------
Requests to log the memory contexts of the backend with the specified process ID. This function can send the request to also auxiliary processes except logger and stats collector.
----------------------

Changed.

As Horiguchi-san told upthread, IMO it's simpler not to use is_aux_proc flag. For example, you can replace this code with

------------------------
proc = BackendPidGetProc(pid);

if (proc != NULL)
backendId = proc->backendId;
else
proc = AuxiliaryPidGetProc(pid);
------------------------

Changed.

PSA v7 patch.

Regards,
Bharath Rupireddy.

Attachments:

v7-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchapplication/octet-stream; name=v7-0001-enhance-pg_log_backend_memory_contexts-to-log-mem.patchDownload
From 77ddb7b2947697e586f8d228e32f683abc157d79 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Fri, 7 Jan 2022 16:48:18 +0000
Subject: [PATCH v7] enhance pg_log_backend_memory_contexts to log memory
 contexts of aux procs

Currently pg_log_backend_memory_contexts() doesn't log the memory
contexts of auxiliary processes such as bgwriter, checkpointer, wal
writer, archiver, startup process and wal receiver. It will be useful
to look at the memory contexts of these processes too, for debugging
purposes and better understanding of the memory usage pattern of
these processes. Inside the code, we could use the AuxiliaryPidGetProc()
to get the PGPROC of these processes. Note that, neither
AuxiliaryPidGetProc() nor BackendPidGetProc() can return PGPROC(as
they don't have PGPROC entries at all) entries for the syslogger,
stats collector processes.
---
 doc/src/sgml/func.sgml                       | 12 ++++----
 src/backend/postmaster/checkpointer.c        |  4 +++
 src/backend/postmaster/interrupt.c           |  5 +++
 src/backend/postmaster/pgarch.c              |  5 +++
 src/backend/postmaster/startup.c             |  5 +++
 src/backend/postmaster/walwriter.c           |  4 +++
 src/backend/utils/adt/mcxtfuncs.c            | 32 +++++++++++++-------
 src/test/regress/expected/misc_functions.out |  6 ++++
 src/test/regress/sql/misc_functions.sql      |  2 ++
 9 files changed, 58 insertions(+), 17 deletions(-)

diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e58efce586..2c17f1d54f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25420,13 +25420,13 @@ SELECT collation for ('foo' COLLATE "de_DE");
         <returnvalue>boolean</returnvalue>
        </para>
        <para>
-        Requests to log the memory contexts of the backend with the
-        specified process ID.  These memory contexts will be logged at
-        <literal>LOG</literal> message level. They will appear in
-        the server log based on the log configuration set
+        Requests to log the memory contexts of the backend with the specified
+        process ID. This function can send the request to all backends and
+        auxiliary processes except logger and stats collector. These memory
+        contexts will be logged at <literal>LOG</literal> message level. They
+        will appear in the server log based on the log configuration set
         (See <xref linkend="runtime-config-logging"/> for more information),
-        but will not be sent to the client regardless of
-        <xref linkend="guc-client-min-messages"/>.
+        but will not be sent to the client regardless of <xref linkend="guc-client-min-messages"/>.
        </para></entry>
       </row>
 
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 25a18b7a14..648522bf83 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -577,6 +577,10 @@ HandleCheckpointerInterrupts(void)
 		/* Normal exit from the checkpointer is here */
 		proc_exit(0);			/* done */
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/interrupt.c b/src/backend/postmaster/interrupt.c
index dd9136a942..7c1d35a715 100644
--- a/src/backend/postmaster/interrupt.c
+++ b/src/backend/postmaster/interrupt.c
@@ -22,6 +22,7 @@
 #include "storage/latch.h"
 #include "storage/procsignal.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 
 volatile sig_atomic_t ConfigReloadPending = false;
 volatile sig_atomic_t ShutdownRequestPending = false;
@@ -43,6 +44,10 @@ HandleMainLoopInterrupts(void)
 
 	if (ShutdownRequestPending)
 		proc_exit(0);
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 /*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 5b6bf9f4e0..6610844998 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -50,6 +50,7 @@
 #include "storage/shmem.h"
 #include "storage/spin.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/ps_status.h"
 
 
@@ -875,4 +876,8 @@ HandlePgArchInterrupts(void)
 		ConfigReloadPending = false;
 		ProcessConfigFile(PGC_SIGHUP);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 47ec737888..2b35a82903 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -32,6 +32,7 @@
 #include "storage/procsignal.h"
 #include "storage/standby.h"
 #include "utils/guc.h"
+#include "utils/memutils.h"
 #include "utils/timeout.h"
 
 
@@ -200,6 +201,10 @@ HandleStartupProcInterrupts(void)
 	/* Process barrier events */
 	if (ProcSignalBarrierPending)
 		ProcessProcSignalBarrier();
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
 
 
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 626fae8454..19c9118fd8 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -306,4 +306,8 @@ HandleWalWriterInterrupts(void)
 
 		proc_exit(0);
 	}
+
+	/* Perform logging of memory contexts of this process */
+	if (LogMemoryContextPending)
+		ProcessLogMemoryContextInterrupt();
 }
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 6ddbf70b30..8e6d21d87c 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -167,26 +167,36 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
  * rate would cause lots of log messages and which can lead to denial of
  * service. Additional roles can be permitted with GRANT.
  *
- * On receipt of this signal, a backend sets the flag in the signal
- * handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
- * memory contexts.
+ * On receipt of this signal, a backend sets the flag in the signal handler,
+ * which causes the next CHECK_FOR_INTERRUPTS() or process specific interrupt
+ * handlers to log the memory contexts.
  */
 Datum
 pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 {
 	int			pid = PG_GETARG_INT32(0);
 	PGPROC	   *proc;
+	BackendId   backendId = InvalidBackendId;
 
 	proc = BackendPidGetProc(pid);
 
 	/*
-	 * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
-	 * we reach kill(), a process for which we get a valid proc here might
-	 * have terminated on its own.  There's no way to acquire a lock on an
-	 * arbitrary process to prevent that. But since this mechanism is usually
-	 * used to debug a backend running and consuming lots of memory, that it
-	 * might end on its own first and its memory contexts are not logged is
-	 * not a problem.
+	 * See if the process with given pid is an auxiliary process. Only regular
+	 * backends will have valid backend id, auxiliary processes don't.
+	 */
+	if (proc != NULL)
+		backendId = proc->backendId;
+	else
+		proc = AuxiliaryPidGetProc(pid);
+
+	/*
+	 * BackendPidGetProc or AuxiliaryPidGetProc returns NULL if the pid isn't
+	 * valid; but by the time we reach kill(), a process for which we get a
+	 * valid proc here might have terminated on its own.  There's no way to
+	 * acquire a lock on an arbitrary process to prevent that. But since this
+	 * mechanism is usually used to debug a backend running and consuming lots
+	 * of memory, that it might end on its own first and its memory contexts
+	 * are not logged is not a problem.
 	 */
 	if (proc == NULL)
 	{
@@ -199,7 +209,7 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
 		PG_RETURN_BOOL(false);
 	}
 
-	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId) < 0)
+	if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
 	{
 		/* Again, just a warning to allow loops */
 		ereport(WARNING,
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 1013d17f87..07c0ffbbe1 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -147,6 +147,12 @@ SELECT pg_log_backend_memory_contexts(pg_backend_pid());
  t
 (1 row)
 
+SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity WHERE backend_type = 'checkpointer';
+ pg_log_backend_memory_contexts 
+--------------------------------
+ t
+(1 row)
+
 CREATE ROLE regress_log_memory;
 SELECT has_function_privilege('regress_log_memory',
   'pg_log_backend_memory_contexts(integer)', 'EXECUTE'); -- no
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 7ab9b2a150..88449edab4 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -41,6 +41,8 @@ SELECT num_nulls();
 
 SELECT pg_log_backend_memory_contexts(pg_backend_pid());
 
+SELECT pg_log_backend_memory_contexts(pid) FROM pg_stat_activity WHERE backend_type = 'checkpointer';
+
 CREATE ROLE regress_log_memory;
 
 SELECT has_function_privilege('regress_log_memory',
-- 
2.25.1

#18Fujii Masao
masao.fujii@oss.nttdata.com
In reply to: Bharath Rupireddy (#17)
Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

On 2022/01/08 1:50, Bharath Rupireddy wrote:

PSA v7 patch.

Thanks for updating the patch!
I applied some cosmetic changes and pushed the patch. Thanks!

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION