Adding clarification to description of IPC wait events XactGroupUpdate and ProcArrayGroupUpdate
Hi,
While preparing for my presentation on PostgreSQL Wait Events at
PgConf India, I was trying to understand *IPC:XactGroupUpdate* in more
detail. PostgreSQL documentation [1]https://www.postgresql.org/docs/current/monitoring-stats.html#WAIT-EVENT-IPC-TABLE mentions:
A process is waiting for the group leader to update the transaction status at the end of a _parallel operation_.
I looked at `TransactionGroupUpdateXidStatus` in PostgreSQL code (`clog.c`)
Line `481` [2]https://github.com/postgres/postgres/blob/master/src/backend/access/transam/clog.c#L481 sets this wait event.
And after reading the code, my understanding is - It does not
necessarily need to be a "_parallel operation_". Or maybe I am just
misinterpreting "parallel operation" in this context. But it is
possible for other users to confuse it with the parallel query (and
parallel workers) feature.
**My understanding is -**
In order to avoid `XactSLRULock` being passed between backends,
backends waiting for it will add themselves to the queue [3]https://github.com/postgres/postgres/blob/master/src/backend/access/transam/clog.c#L399. The
first backend in the queue (also the leader) will be the only one to
acquire `XactSLRULock` and update the XID status for all those pids
which are in the queue. This IPC wait event (`XactGroupUpdate`) is
observed in other backened processes who are in the queue, waiting for
the group leader to update the XID status.
We can add more clarity on what this wait event means. A similar
change should be done for `ProcArrayGroupUpdate` to indicate that the
wait event is a result of concurrent backend processes trying to clear
the transaction id (instead of saying "parallel operation”).
I am attaching a patch for consideration. This should also be
backpatched-through version 13.
[1]: https://www.postgresql.org/docs/current/monitoring-stats.html#WAIT-EVENT-IPC-TABLE
[2]: https://github.com/postgres/postgres/blob/master/src/backend/access/transam/clog.c#L481
[3]: https://github.com/postgres/postgres/blob/master/src/backend/access/transam/clog.c#L399
Thanks,
Sameer
DB Specialist,
Amazon Web Services
Attachments:
v1-0001-Update-the-description-of-IPC-wait-events-XactGro.patchapplication/octet-stream; name=v1-0001-Update-the-description-of-IPC-wait-events-XactGro.patchDownload
From fff9d783ac3013f56cc00a86b1d3023729060b03 Mon Sep 17 00:00:00 2001
From: Sameer Kumar <kusme@amazon.com>
Date: Tue, 23 Jul 2024 13:30:41 +0800
Subject: [PATCH v1] Update the description of IPC wait events XactGroupUpdate
and ProcArrayGroupUpdate to avoid confusion with parallel workers and
parallel query/parallel workers.
---
src/backend/utils/activity/wait_event_names.txt | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index db37beeaae..5872d27467 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -142,7 +142,7 @@ MULTIXACT_CREATION "Waiting for a multixact creation to complete."
PARALLEL_BITMAP_SCAN "Waiting for parallel bitmap scan to become initialized."
PARALLEL_CREATE_INDEX_SCAN "Waiting for parallel <command>CREATE INDEX</command> workers to finish heap scan."
PARALLEL_FINISH "Waiting for parallel workers to finish computing."
-PROCARRAY_GROUP_UPDATE "Waiting for the group leader to clear the transaction ID at end of a parallel operation."
+PROCARRAY_GROUP_UPDATE "Waiting for the group leader process to clear the transaction ID of this backend at the end of a transaction."
PROC_SIGNAL_BARRIER "Waiting for a barrier event to be processed by all backends."
PROMOTE "Waiting for standby promotion."
RECOVERY_CONFLICT_SNAPSHOT "Waiting for recovery conflict resolution for a vacuum cleanup."
@@ -157,7 +157,7 @@ SYNC_REP "Waiting for confirmation from a remote server during synchronous repli
WAL_RECEIVER_EXIT "Waiting for the WAL receiver to exit."
WAL_RECEIVER_WAIT_START "Waiting for startup process to send initial data for streaming replication."
WAL_SUMMARY_READY "Waiting for a new WAL summary to be generated."
-XACT_GROUP_UPDATE "Waiting for the group leader to update transaction status at end of a parallel operation."
+XACT_GROUP_UPDATE "Waiting for the group leader process to update the transaction status for this backend."
ABI_compatibility:
--
2.40.1
On Thu, Jul 25, 2024 at 11:13:39AM +0800, SAMEER KUMAR wrote:
While preparing for my presentation on PostgreSQL Wait Events at
PgConf India, I was trying to understand *IPC:XactGroupUpdate* in more
detail. PostgreSQL documentation [1] mentions:A process is waiting for the group leader to update the transaction status at the end of a _parallel operation_.
I looked at `TransactionGroupUpdateXidStatus` in PostgreSQL code (`clog.c`)
Line `481` [2] sets this wait event.And after reading the code, my understanding is - It does not
necessarily need to be a "_parallel operation_". Or maybe I am just
misinterpreting "parallel operation" in this context. But it is
possible for other users to confuse it with the parallel query (and
parallel workers) feature.[...]
We can add more clarity on what this wait event means. A similar
change should be done for `ProcArrayGroupUpdate` to indicate that the
wait event is a result of concurrent backend processes trying to clear
the transaction id (instead of saying "parallel operation").
Both of these wait events had descriptions similar to what you are
proposing when they were first introduced (commits d4116a7 and baaf272),
but they were changed to the current wording by commit 3048898. I skimmed
through the thread for the latter commit [0]/messages/by-id/21247.1589296570@sss.pgh.pa.us but didn't see anything that
explained why it was changed.
[0]: /messages/by-id/21247.1589296570@sss.pgh.pa.us
--
nathan
Thanks for responding.
On Wed, Aug 14, 2024 at 10:29 PM Nathan Bossart <nathandbossart@gmail.com>
wrote:
On Thu, Jul 25, 2024 at 11:13:39AM +0800, SAMEER KUMAR wrote:
While preparing for my presentation on PostgreSQL Wait Events at
PgConf India, I was trying to understand *IPC:XactGroupUpdate* in more
detail. PostgreSQL documentation [1] mentions:A process is waiting for the group leader to update the transaction
status at the end of a _parallel operation_.
I looked at `TransactionGroupUpdateXidStatus` in PostgreSQL code
(`clog.c`)
Line `481` [2] sets this wait event.
And after reading the code, my understanding is - It does not
necessarily need to be a "_parallel operation_". Or maybe I am just
misinterpreting "parallel operation" in this context. But it is
possible for other users to confuse it with the parallel query (and
parallel workers) feature.[...]
We can add more clarity on what this wait event means. A similar
change should be done for `ProcArrayGroupUpdate` to indicate that the
wait event is a result of concurrent backend processes trying to clear
the transaction id (instead of saying "parallel operation").Both of these wait events had descriptions similar to what you are
proposing when they were first introduced (commits d4116a7 and baaf272),
but they were changed to the current wording by commit 3048898. I skimmed
through the thread for the latter commit [0] but didn't see anything that
explained why it was changed.
Yes, while reviewing the history of changes, I too noticed the same. The
documentation of older versions (v12 [1]https://www.postgresql.org/docs/12/monitoring-stats.html#WAIT-EVENT-TABLE) still has old descriptions.
[0] /messages/by-id/21247.1589296570@sss.pgh.pa.us
--
nathan
[1]: https://www.postgresql.org/docs/12/monitoring-stats.html#WAIT-EVENT-TABLE
https://www.postgresql.org/docs/12/monitoring-stats.html#WAIT-EVENT-TABLE
On Wed, Aug 14, 2024 at 10:38:49PM +0800, SAMEER KUMAR wrote:
Yes, while reviewing the history of changes, I too noticed the same. The
documentation of older versions (v12 [1]) still has old descriptions.
After reading the related threads and code, I'm inclined to agree that this
is a mistake, or at least that the current wording is likely to mislead
folks into thinking it has something to do with parallel query. I noticed
that your patch changed a few things in the description, but IMHO we should
keep the fix focused, i.e., just replace "end of a parallel operation" with
"transaction end." I've attached a modified version of the patch with this
change.
--
nathan
Attachments:
v2-0001-Fix-a-couple-of-wait-event-descriptions.patchtext/plain; charset=us-asciiDownload
From e23d495cae2d48e5d52810afff083e2e94df786b Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Wed, 14 Aug 2024 14:44:01 -0500
Subject: [PATCH v2 1/1] Fix a couple of wait event descriptions.
Author: Sameer Kumar
Discussion: https://postgr.es/m/CAGPeHmh6UMrKQHKCmX%2B5vV5TH9P%3DKw9en3k68qEem6J%3DyrZPUA%40mail.gmail.com
Backpatch-through: 13
---
src/backend/utils/activity/wait_event_names.txt | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index d10ca723dc..8efb4044d6 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -143,7 +143,7 @@ MULTIXACT_CREATION "Waiting for a multixact creation to complete."
PARALLEL_BITMAP_SCAN "Waiting for parallel bitmap scan to become initialized."
PARALLEL_CREATE_INDEX_SCAN "Waiting for parallel <command>CREATE INDEX</command> workers to finish heap scan."
PARALLEL_FINISH "Waiting for parallel workers to finish computing."
-PROCARRAY_GROUP_UPDATE "Waiting for the group leader to clear the transaction ID at end of a parallel operation."
+PROCARRAY_GROUP_UPDATE "Waiting for the group leader to clear the transaction ID at transaction end."
PROC_SIGNAL_BARRIER "Waiting for a barrier event to be processed by all backends."
PROMOTE "Waiting for standby promotion."
RECOVERY_CONFLICT_SNAPSHOT "Waiting for recovery conflict resolution for a vacuum cleanup."
@@ -158,7 +158,7 @@ SYNC_REP "Waiting for confirmation from a remote server during synchronous repli
WAL_RECEIVER_EXIT "Waiting for the WAL receiver to exit."
WAL_RECEIVER_WAIT_START "Waiting for startup process to send initial data for streaming replication."
WAL_SUMMARY_READY "Waiting for a new WAL summary to be generated."
-XACT_GROUP_UPDATE "Waiting for the group leader to update transaction status at end of a parallel operation."
+XACT_GROUP_UPDATE "Waiting for the group leader to update transaction status at transaction end."
ABI_compatibility:
--
2.39.3 (Apple Git-146)
On Thu, Aug 15, 2024 at 4:00 AM Nathan Bossart <nathandbossart@gmail.com>
wrote:
On Wed, Aug 14, 2024 at 10:38:49PM +0800, SAMEER KUMAR wrote:
Yes, while reviewing the history of changes, I too noticed the same. The
documentation of older versions (v12 [1]) still has old descriptions.After reading the related threads and code, I'm inclined to agree that this
is a mistake, or at least that the current wording is likely to mislead
folks into thinking it has something to do with parallel query. I noticed
that your patch changed a few things in the description, but IMHO we should
keep the fix focused, i.e., just replace "end of a parallel operation" with
"transaction end." I've attached a modified version of the patch with this
change.
Thanks for the feedback Nathan.
I think it is important to indicate that the group leader is responsible
for clearing the transaction ID/transaction status of other backends
(including this one).
If you suggest that we keep it simple, I don't see any other issues with
your patch.
Show quoted text
--
nathan
On Thu, Aug 15, 2024 at 11:25:25AM +0800, SAMEER KUMAR wrote:
I think it is important to indicate that the group leader is responsible
for clearing the transaction ID/transaction status of other backends
(including this one).
Your proposal is
Waiting for the group leader process to clear the transaction ID of
this backend at the end of a transaction.
Waiting for the group leader process to update the transaction status
for this backend.
Mine is
Waiting for the group leader to clear the transaction ID at transaction
end.
Waiting for the group leader to update transaction status at
transaction end.
IMHO the latter doesn't convey substantially less information, and it fits
a little better with the terse style of the other wait events nearby. But
I'll yield to majority opinion here.
--
nathan
On Thu, Aug 15, 2024 at 8:11 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
On Thu, Aug 15, 2024 at 11:25:25AM +0800, SAMEER KUMAR wrote:
I think it is important to indicate that the group leader is responsible
for clearing the transaction ID/transaction status of other backends
(including this one).Your proposal is
Waiting for the group leader process to clear the transaction ID of
this backend at the end of a transaction.Waiting for the group leader process to update the transaction status
for this backend.Mine is
Waiting for the group leader to clear the transaction ID at transaction
end.Waiting for the group leader to update transaction status at
transaction end.IMHO the latter doesn't convey substantially less information, and it fits
a little better with the terse style of the other wait events nearby.
+1 for Nathan's version. It is quite close to the previous version,
for which we haven't heard any complaints since they were introduced.
--
With Regards,
Amit Kapila.
On Tue, Aug 20, 2024 at 02:12:25PM +0530, Amit Kapila wrote:
+1 for Nathan's version. It is quite close to the previous version,
for which we haven't heard any complaints since they were introduced.
Committed, thanks.
--
nathan