Add memory_limit_hits to pg_stat_replication_slots
Hi hackers,
I think that it's currently not always possible to determine how many times
logical_decoding_work_mem has been reached.
For example, say a transaction is made of 40 subtransactions, and I get:
slot_name | spill_txns | spill_count | total_txns
--------------+------------+-------------+------------
logical_slot | 40 | 41 | 1
(1 row)
Then I know that logical_decoding_work_mem has been reached one time (total_txns).
But as soon as another transaction is decoded (that does not involve spilling):
slot_name | spill_txns | spill_count | total_txns
--------------+------------+-------------+------------
logical_slot | 40 | 41 | 2
(1 row)
Then we don't know if logical_decoding_work_mem has been reached one or two
times.
Please find attached a patch to $SUBJECT, to report the number of times the
logical_decoding_work_mem has been reached.
With such a counter one could get a ratio like total_txns/memory_limit_hits.
That could help to see if reaching logical_decoding_work_mem is rare or
frequent enough. If frequent, then maybe there is a need to adjust
logical_decoding_work_mem.
Based on my simple example above, one could say that it might be possible to get
the same with:
(spill_count - spill_txns) + (stream_count - stream_txns)
but that doesn't appear to be the case with a more complicated example (277 vs 247):
slot_name | spill_txns | spill_count | total_txns | stream_txns | stream_count | memory_limit_hits | (spc-spct)+(strc-strt)
--------------+------------+-------------+------------+-------------+--------------+-------------------+------------------------
logical_slot | 405 | 552 | 19 | 5 | 105 | 277 | 247
(1 row)
Not sure I like memory_limit_hits that much, maybe work_mem_exceeded is better?
Looking forward to your feedback,
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v1-0001-Add-memory_limit_hits-to-pg_stat_replication_slot.patchtext/x-diff; charset=us-asciiDownload
From e959e12e7809e6709d5f22ae655364dd6f294ebb Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Tue, 26 Aug 2025 13:08:13 +0000
Subject: [PATCH v1] Add memory_limit_hits to pg_stat_replication_slots
It's currently not always possible to determine how many times the
logical_decoding_work_mem has been reached.
So adding a new counter, memory_limit_hits to report the number of times the
logical_decoding_work_mem has been reached while decoding.
With such a counter one could get a ratio like total_txns/memory_limit_hits.
That could help to see if reaching logical_decoding_work_mem is rare or
frequent enough. If frequent, then maybe there is a need to adjust
logical_decoding_work_mem.
XXXX: Bump catversion.
---
contrib/test_decoding/expected/stats.out | 68 +++++++++----------
contrib/test_decoding/sql/stats.sql | 10 +--
doc/src/sgml/monitoring.sgml | 11 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/replication/logical/logical.c | 7 +-
.../replication/logical/reorderbuffer.c | 7 +-
src/backend/utils/activity/pgstat_replslot.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 11 +--
src/include/catalog/pg_proc.dat | 6 +-
src/include/pgstat.h | 1 +
src/include/replication/reorderbuffer.h | 3 +
src/test/regress/expected/rules.out | 3 +-
12 files changed, 79 insertions(+), 50 deletions(-)
47.7% contrib/test_decoding/expected/
16.9% contrib/test_decoding/sql/
6.7% doc/src/sgml/
8.3% src/backend/replication/logical/
9.7% src/backend/utils/adt/
3.7% src/include/catalog/
3.1% src/test/regress/expected/
3.5% src/
diff --git a/contrib/test_decoding/expected/stats.out b/contrib/test_decoding/expected/stats.out
index de6dc416130..642ebe1ee5e 100644
--- a/contrib/test_decoding/expected/stats.out
+++ b/contrib/test_decoding/expected/stats.out
@@ -37,12 +37,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | t | t
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | memory_limit_hits
+------------------------+------------+-------------+------------+-------------+-------------------
+ regression_slot_stats1 | t | t | t | t | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
RESET logical_decoding_work_mem;
@@ -53,12 +53,12 @@ SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | memory_limit_hits
+------------------------+------------+-------------+------------+-------------+-------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
-- reset stats for all slots
@@ -68,27 +68,27 @@ SELECT pg_stat_reset_replication_slot(NULL);
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | f | f
- regression_slot_stats3 | t | t | f | f
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | memory_limit_hits
+------------------------+------------+-------------+------------+-------------+-------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | f | f | t
+ regression_slot_stats3 | t | t | f | f | t
(3 rows)
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | memory_limit_hits | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
SELECT pg_stat_reset_replication_slot('do-not-exist');
ERROR: replication slot "do-not-exist" does not exist
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | memory_limit_hits | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
-- spilling the xact
@@ -110,12 +110,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
- slot_name | spill_txns | spill_count
-------------------------+------------+-------------
- regression_slot_stats1 | t | t
- regression_slot_stats2 | f | f
- regression_slot_stats3 | f | f
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, memory_limit_hits > 0 AS memory_limit_hits FROM pg_stat_replication_slots;
+ slot_name | spill_txns | spill_count | memory_limit_hits
+------------------------+------------+-------------+-------------------
+ regression_slot_stats1 | t | t | t
+ regression_slot_stats2 | f | f | f
+ regression_slot_stats3 | f | f | f
(3 rows)
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
@@ -165,10 +165,10 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
- slot_name | spill_txns | spill_count
----------------------------------+------------+-------------
- regression_slot_stats4_twophase | 0 | 0
+SELECT slot_name, spill_txns, spill_count, memory_limit_hits FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+ slot_name | spill_txns | spill_count | memory_limit_hits
+---------------------------------+------------+-------------+-------------------
+ regression_slot_stats4_twophase | 0 | 0 | 0
(1 row)
DROP TABLE stats_test;
diff --git a/contrib/test_decoding/sql/stats.sql b/contrib/test_decoding/sql/stats.sql
index a022fe1bf07..8a58d05a764 100644
--- a/contrib/test_decoding/sql/stats.sql
+++ b/contrib/test_decoding/sql/stats.sql
@@ -15,16 +15,16 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats1', NULL,
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats2', NULL, NULL, 'skip-empty-xacts', '1');
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats3', NULL, NULL, 'skip-empty-xacts', '1');
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
RESET logical_decoding_work_mem;
-- reset stats for one slot, others should be unaffected
SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
-- reset stats for all slots
SELECT pg_stat_reset_replication_slot(NULL);
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
@@ -41,7 +41,7 @@ SELECT count(*) FROM pg_logical_slot_peek_changes('regression_slot_stats1', NULL
-- background transaction (say by autovacuum) happens in parallel to the main
-- transaction.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, memory_limit_hits > 0 AS memory_limit_hits FROM pg_stat_replication_slots;
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
-- https://postgr.es/m/20210317230447.c7uc4g3vbs4wi32i%40alap3.anarazel.de
@@ -65,7 +65,7 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats4_twophas
-- Verify that the decoding doesn't spill already-aborted transaction's changes.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+SELECT slot_name, spill_txns, spill_count, memory_limit_hits FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
DROP TABLE stats_test;
SELECT pg_drop_replication_slot('regression_slot_stats1'),
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 3f4a27a736e..3b175a1d3e1 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1644,6 +1644,17 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>memory_limit_hits</structfield><type>bigint</type>
+ </para>
+ <para>
+ Number of times <literal>logical_decoding_work_mem</literal> has been
+ reached while decoding changes from WAL for this slot.
+ </para>
+ </entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>stats_reset</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1b3c5a55882..fbbb963734a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1061,6 +1061,7 @@ CREATE VIEW pg_stat_replication_slots AS
s.stream_bytes,
s.total_txns,
s.total_bytes,
+ s.memory_limit_hits,
s.stats_reset
FROM pg_replication_slots as r,
LATERAL pg_stat_get_replication_slot(slot_name) as s
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 7e363a7c05b..d879c183858 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1958,7 +1958,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
if (rb->spillBytes <= 0 && rb->streamBytes <= 0 && rb->totalBytes <= 0)
return;
- elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
+ elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
rb,
rb->spillTxns,
rb->spillCount,
@@ -1967,7 +1967,8 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamCount,
rb->streamBytes,
rb->totalTxns,
- rb->totalBytes);
+ rb->totalBytes,
+ rb->memory_limit_hits);
repSlotStat.spill_txns = rb->spillTxns;
repSlotStat.spill_count = rb->spillCount;
@@ -1977,6 +1978,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
repSlotStat.stream_bytes = rb->streamBytes;
repSlotStat.total_txns = rb->totalTxns;
repSlotStat.total_bytes = rb->totalBytes;
+ repSlotStat.memory_limit_hits = rb->memory_limit_hits;
pgstat_report_replslot(ctx->slot, &repSlotStat);
@@ -1988,6 +1990,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamBytes = 0;
rb->totalTxns = 0;
rb->totalBytes = 0;
+ rb->memory_limit_hits = 0;
}
/*
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 34cf05668ae..e8e8cf39b97 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -392,6 +392,7 @@ ReorderBufferAllocate(void)
buffer->streamBytes = 0;
buffer->totalTxns = 0;
buffer->totalBytes = 0;
+ buffer->memory_limit_hits = 0;
buffer->current_restart_decoding_lsn = InvalidXLogRecPtr;
@@ -3883,13 +3884,17 @@ static void
ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
{
ReorderBufferTXN *txn;
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024);
+
+ if (memory_limit_reached)
+ rb->memory_limit_hits += 1;
/*
* Bail out if debug_logical_replication_streaming is buffered and we
* haven't exceeded the memory limit.
*/
if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
- rb->size < logical_decoding_work_mem * (Size) 1024)
+ !memory_limit_reached)
return;
/*
diff --git a/src/backend/utils/activity/pgstat_replslot.c b/src/backend/utils/activity/pgstat_replslot.c
index ccfb11c49bf..27e9384f590 100644
--- a/src/backend/utils/activity/pgstat_replslot.c
+++ b/src/backend/utils/activity/pgstat_replslot.c
@@ -96,6 +96,7 @@ pgstat_report_replslot(ReplicationSlot *slot, const PgStat_StatReplSlotEntry *re
REPLSLOT_ACC(stream_bytes);
REPLSLOT_ACC(total_txns);
REPLSLOT_ACC(total_bytes);
+ REPLSLOT_ACC(memory_limit_hits);
#undef REPLSLOT_ACC
pgstat_unlock_entry(entry_ref);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..b400794e9f8 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2100,7 +2100,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
Datum
pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_REPLICATION_SLOT_COLS 10
+#define PG_STAT_GET_REPLICATION_SLOT_COLS 11
text *slotname_text = PG_GETARG_TEXT_P(0);
NameData slotname;
TupleDesc tupdesc;
@@ -2129,7 +2129,9 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_bytes",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 10, "stats_reset",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 10, "memory_limit_hits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 11, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
@@ -2154,11 +2156,12 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
values[6] = Int64GetDatum(slotent->stream_bytes);
values[7] = Int64GetDatum(slotent->total_txns);
values[8] = Int64GetDatum(slotent->total_bytes);
+ values[9] = Int64GetDatum(slotent->memory_limit_hits);
if (slotent->stat_reset_timestamp == 0)
- nulls[9] = true;
+ nulls[10] = true;
else
- values[9] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
+ values[10] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 118d6da1ace..991d1982c7c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5675,9 +5675,9 @@
{ oid => '6169', descr => 'statistics: information about replication slot',
proname => 'pg_stat_get_replication_slot', provolatile => 's',
proparallel => 'r', prorettype => 'record', proargtypes => 'text',
- proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
+ proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,memory_limit_hits,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
{ oid => '6230', descr => 'statistics: check if a stats object exists',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 202bd2d5ace..e361412bf60 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -395,6 +395,7 @@ typedef struct PgStat_StatReplSlotEntry
PgStat_Counter stream_bytes;
PgStat_Counter total_txns;
PgStat_Counter total_bytes;
+ PgStat_Counter memory_limit_hits;
TimestampTz stat_reset_timestamp;
} PgStat_StatReplSlotEntry;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index fa0745552f8..b30d90c2013 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -696,6 +696,9 @@ struct ReorderBuffer
*/
int64 totalTxns; /* total number of transactions sent */
int64 totalBytes; /* total amount of data decoded */
+
+ /* Number of times logical_decoding_work_mem has been reached */
+ int64 memory_limit_hits;
};
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..8ce27032b87 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2140,9 +2140,10 @@ pg_stat_replication_slots| SELECT s.slot_name,
s.stream_bytes,
s.total_txns,
s.total_bytes,
+ s.memory_limit_hits,
s.stats_reset
FROM pg_replication_slots r,
- LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, total_txns, total_bytes, stats_reset)
+ LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, total_txns, total_bytes, memory_limit_hits, stats_reset)
WHERE (r.datoid IS NOT NULL);
pg_stat_slru| SELECT name,
blks_zeroed,
--
2.34.1
On Wed, Aug 27, 2025 at 12:26 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
Hi hackers,
I think that it's currently not always possible to determine how many times
logical_decoding_work_mem has been reached.For example, say a transaction is made of 40 subtransactions, and I get:
slot_name | spill_txns | spill_count | total_txns
--------------+------------+-------------+------------
logical_slot | 40 | 41 | 1
(1 row)Then I know that logical_decoding_work_mem has been reached one time (total_txns).
But as soon as another transaction is decoded (that does not involve spilling):
slot_name | spill_txns | spill_count | total_txns
--------------+------------+-------------+------------
logical_slot | 40 | 41 | 2
(1 row)Then we don't know if logical_decoding_work_mem has been reached one or two
times.Please find attached a patch to $SUBJECT, to report the number of times the
logical_decoding_work_mem has been reached.With such a counter one could get a ratio like total_txns/memory_limit_hits.
That could help to see if reaching logical_decoding_work_mem is rare or
frequent enough. If frequent, then maybe there is a need to adjust
logical_decoding_work_mem.Based on my simple example above, one could say that it might be possible to get
the same with:(spill_count - spill_txns) + (stream_count - stream_txns)
but that doesn't appear to be the case with a more complicated example (277 vs 247):
slot_name | spill_txns | spill_count | total_txns | stream_txns | stream_count | memory_limit_hits | (spc-spct)+(strc-strt)
--------------+------------+-------------+------------+-------------+--------------+-------------------+------------------------
logical_slot | 405 | 552 | 19 | 5 | 105 | 277 | 247
(1 row)Not sure I like memory_limit_hits that much, maybe work_mem_exceeded is better?
Looking forward to your feedback,
Yes, it's a quite different situation in two cases: spilling 100
transactions in one ReorderBufferCheckMemoryLimit() call and spilling
1 transaction in each 100 ReorderBufferCheckMemoryLimit() calls, even
though spill_txn is 100 in both cases. And we don't have any
statistics to distinguish between these cases. I agree with the
statistics.
One minor comment is:
@@ -1977,6 +1978,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
repSlotStat.stream_bytes = rb->streamBytes;
repSlotStat.total_txns = rb->totalTxns;
repSlotStat.total_bytes = rb->totalBytes;
+ repSlotStat.memory_limit_hits = rb->memory_limit_hits;Since other statistics counter names are camel cases I think it's
better to follow that for the new counter.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Hi,
On Thu, Sep 11, 2025 at 03:24:54PM -0700, Masahiko Sawada wrote:
On Wed, Aug 27, 2025 at 12:26 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Looking forward to your feedback,
Yes,
Thanks for looking at it!
it's a quite different situation in two cases: spilling 100
transactions in one ReorderBufferCheckMemoryLimit() call and spilling
1 transaction in each 100 ReorderBufferCheckMemoryLimit() calls, even
though spill_txn is 100 in both cases. And we don't have any
statistics to distinguish between these cases.
Right.
One minor comment is:
@@ -1977,6 +1978,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx) repSlotStat.stream_bytes = rb->streamBytes; repSlotStat.total_txns = rb->totalTxns; repSlotStat.total_bytes = rb->totalBytes; + repSlotStat.memory_limit_hits = rb->memory_limit_hits;Since other statistics counter names are camel cases I think it's
better to follow that for the new counter.
Makes sense, done with memoryLimitHits in v2 attached (that's the only change
as compared with v1).
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v2-0001-Add-memory_limit_hits-to-pg_stat_replication_slot.patchtext/x-diff; charset=us-asciiDownload
From 111575015f513f4b2678a84eee387798ea6e5969 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Tue, 26 Aug 2025 13:08:13 +0000
Subject: [PATCH v2] Add memory_limit_hits to pg_stat_replication_slots
It's currently not always possible to determine how many times the
logical_decoding_work_mem has been reached.
So adding a new counter, memory_limit_hits to report the number of times the
logical_decoding_work_mem has been reached while decoding.
With such a counter one could get a ratio like total_txns/memory_limit_hits.
That could help to see if reaching logical_decoding_work_mem is rare or
frequent enough. If frequent, then maybe there is a need to adjust
logical_decoding_work_mem.
XXXX: Bump catversion.
---
contrib/test_decoding/expected/stats.out | 68 +++++++++----------
contrib/test_decoding/sql/stats.sql | 10 +--
doc/src/sgml/monitoring.sgml | 11 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/replication/logical/logical.c | 7 +-
.../replication/logical/reorderbuffer.c | 7 +-
src/backend/utils/activity/pgstat_replslot.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 11 +--
src/include/catalog/pg_proc.dat | 6 +-
src/include/pgstat.h | 1 +
src/include/replication/reorderbuffer.h | 3 +
src/test/regress/expected/rules.out | 3 +-
12 files changed, 79 insertions(+), 50 deletions(-)
47.8% contrib/test_decoding/expected/
16.9% contrib/test_decoding/sql/
6.7% doc/src/sgml/
8.2% src/backend/replication/logical/
9.7% src/backend/utils/adt/
3.7% src/include/catalog/
3.1% src/test/regress/expected/
3.5% src/
diff --git a/contrib/test_decoding/expected/stats.out b/contrib/test_decoding/expected/stats.out
index de6dc416130..642ebe1ee5e 100644
--- a/contrib/test_decoding/expected/stats.out
+++ b/contrib/test_decoding/expected/stats.out
@@ -37,12 +37,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | t | t
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | memory_limit_hits
+------------------------+------------+-------------+------------+-------------+-------------------
+ regression_slot_stats1 | t | t | t | t | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
RESET logical_decoding_work_mem;
@@ -53,12 +53,12 @@ SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | memory_limit_hits
+------------------------+------------+-------------+------------+-------------+-------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
-- reset stats for all slots
@@ -68,27 +68,27 @@ SELECT pg_stat_reset_replication_slot(NULL);
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | f | f
- regression_slot_stats3 | t | t | f | f
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | memory_limit_hits
+------------------------+------------+-------------+------------+-------------+-------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | f | f | t
+ regression_slot_stats3 | t | t | f | f | t
(3 rows)
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | memory_limit_hits | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
SELECT pg_stat_reset_replication_slot('do-not-exist');
ERROR: replication slot "do-not-exist" does not exist
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | memory_limit_hits | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
-- spilling the xact
@@ -110,12 +110,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
- slot_name | spill_txns | spill_count
-------------------------+------------+-------------
- regression_slot_stats1 | t | t
- regression_slot_stats2 | f | f
- regression_slot_stats3 | f | f
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, memory_limit_hits > 0 AS memory_limit_hits FROM pg_stat_replication_slots;
+ slot_name | spill_txns | spill_count | memory_limit_hits
+------------------------+------------+-------------+-------------------
+ regression_slot_stats1 | t | t | t
+ regression_slot_stats2 | f | f | f
+ regression_slot_stats3 | f | f | f
(3 rows)
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
@@ -165,10 +165,10 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
- slot_name | spill_txns | spill_count
----------------------------------+------------+-------------
- regression_slot_stats4_twophase | 0 | 0
+SELECT slot_name, spill_txns, spill_count, memory_limit_hits FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+ slot_name | spill_txns | spill_count | memory_limit_hits
+---------------------------------+------------+-------------+-------------------
+ regression_slot_stats4_twophase | 0 | 0 | 0
(1 row)
DROP TABLE stats_test;
diff --git a/contrib/test_decoding/sql/stats.sql b/contrib/test_decoding/sql/stats.sql
index a022fe1bf07..8a58d05a764 100644
--- a/contrib/test_decoding/sql/stats.sql
+++ b/contrib/test_decoding/sql/stats.sql
@@ -15,16 +15,16 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats1', NULL,
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats2', NULL, NULL, 'skip-empty-xacts', '1');
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats3', NULL, NULL, 'skip-empty-xacts', '1');
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
RESET logical_decoding_work_mem;
-- reset stats for one slot, others should be unaffected
SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
-- reset stats for all slots
SELECT pg_stat_reset_replication_slot(NULL);
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, memory_limit_hits = 0 AS memory_limit_hits FROM pg_stat_replication_slots ORDER BY slot_name;
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
@@ -41,7 +41,7 @@ SELECT count(*) FROM pg_logical_slot_peek_changes('regression_slot_stats1', NULL
-- background transaction (say by autovacuum) happens in parallel to the main
-- transaction.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, memory_limit_hits > 0 AS memory_limit_hits FROM pg_stat_replication_slots;
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
-- https://postgr.es/m/20210317230447.c7uc4g3vbs4wi32i%40alap3.anarazel.de
@@ -65,7 +65,7 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats4_twophas
-- Verify that the decoding doesn't spill already-aborted transaction's changes.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+SELECT slot_name, spill_txns, spill_count, memory_limit_hits FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
DROP TABLE stats_test;
SELECT pg_drop_replication_slot('regression_slot_stats1'),
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 3f4a27a736e..3b175a1d3e1 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1644,6 +1644,17 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>memory_limit_hits</structfield><type>bigint</type>
+ </para>
+ <para>
+ Number of times <literal>logical_decoding_work_mem</literal> has been
+ reached while decoding changes from WAL for this slot.
+ </para>
+ </entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>stats_reset</structfield> <type>timestamp with time zone</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index c77fa0234bb..8a2b9eec340 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1061,6 +1061,7 @@ CREATE VIEW pg_stat_replication_slots AS
s.stream_bytes,
s.total_txns,
s.total_bytes,
+ s.memory_limit_hits,
s.stats_reset
FROM pg_replication_slots as r,
LATERAL pg_stat_get_replication_slot(slot_name) as s
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c68c0481f42..fdc0887caae 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1958,7 +1958,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
if (rb->spillBytes <= 0 && rb->streamBytes <= 0 && rb->totalBytes <= 0)
return;
- elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
+ elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
rb,
rb->spillTxns,
rb->spillCount,
@@ -1967,7 +1967,8 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamCount,
rb->streamBytes,
rb->totalTxns,
- rb->totalBytes);
+ rb->totalBytes,
+ rb->memoryLimitHits);
repSlotStat.spill_txns = rb->spillTxns;
repSlotStat.spill_count = rb->spillCount;
@@ -1977,6 +1978,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
repSlotStat.stream_bytes = rb->streamBytes;
repSlotStat.total_txns = rb->totalTxns;
repSlotStat.total_bytes = rb->totalBytes;
+ repSlotStat.memory_limit_hits = rb->memoryLimitHits;
pgstat_report_replslot(ctx->slot, &repSlotStat);
@@ -1988,6 +1990,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamBytes = 0;
rb->totalTxns = 0;
rb->totalBytes = 0;
+ rb->memoryLimitHits = 0;
}
/*
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 4736f993c37..1775da5d4be 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -392,6 +392,7 @@ ReorderBufferAllocate(void)
buffer->streamBytes = 0;
buffer->totalTxns = 0;
buffer->totalBytes = 0;
+ buffer->memoryLimitHits = 0;
buffer->current_restart_decoding_lsn = InvalidXLogRecPtr;
@@ -3898,13 +3899,17 @@ static void
ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
{
ReorderBufferTXN *txn;
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024);
+
+ if (memory_limit_reached)
+ rb->memoryLimitHits += 1;
/*
* Bail out if debug_logical_replication_streaming is buffered and we
* haven't exceeded the memory limit.
*/
if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
- rb->size < logical_decoding_work_mem * (Size) 1024)
+ !memory_limit_reached)
return;
/*
diff --git a/src/backend/utils/activity/pgstat_replslot.c b/src/backend/utils/activity/pgstat_replslot.c
index ccfb11c49bf..27e9384f590 100644
--- a/src/backend/utils/activity/pgstat_replslot.c
+++ b/src/backend/utils/activity/pgstat_replslot.c
@@ -96,6 +96,7 @@ pgstat_report_replslot(ReplicationSlot *slot, const PgStat_StatReplSlotEntry *re
REPLSLOT_ACC(stream_bytes);
REPLSLOT_ACC(total_txns);
REPLSLOT_ACC(total_bytes);
+ REPLSLOT_ACC(memory_limit_hits);
#undef REPLSLOT_ACC
pgstat_unlock_entry(entry_ref);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..b400794e9f8 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2100,7 +2100,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
Datum
pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_REPLICATION_SLOT_COLS 10
+#define PG_STAT_GET_REPLICATION_SLOT_COLS 11
text *slotname_text = PG_GETARG_TEXT_P(0);
NameData slotname;
TupleDesc tupdesc;
@@ -2129,7 +2129,9 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_bytes",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 10, "stats_reset",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 10, "memory_limit_hits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 11, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
@@ -2154,11 +2156,12 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
values[6] = Int64GetDatum(slotent->stream_bytes);
values[7] = Int64GetDatum(slotent->total_txns);
values[8] = Int64GetDatum(slotent->total_bytes);
+ values[9] = Int64GetDatum(slotent->memory_limit_hits);
if (slotent->stat_reset_timestamp == 0)
- nulls[9] = true;
+ nulls[10] = true;
else
- values[9] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
+ values[10] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 01eba3b5a19..1a2d213408b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5687,9 +5687,9 @@
{ oid => '6169', descr => 'statistics: information about replication slot',
proname => 'pg_stat_get_replication_slot', provolatile => 's',
proparallel => 'r', prorettype => 'record', proargtypes => 'text',
- proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
+ proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,memory_limit_hits,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
{ oid => '6230', descr => 'statistics: check if a stats object exists',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f402b17295c..9215027d0ac 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -395,6 +395,7 @@ typedef struct PgStat_StatReplSlotEntry
PgStat_Counter stream_bytes;
PgStat_Counter total_txns;
PgStat_Counter total_bytes;
+ PgStat_Counter memory_limit_hits;
TimestampTz stat_reset_timestamp;
} PgStat_StatReplSlotEntry;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index fa0745552f8..89ac782bfad 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -696,6 +696,9 @@ struct ReorderBuffer
*/
int64 totalTxns; /* total number of transactions sent */
int64 totalBytes; /* total amount of data decoded */
+
+ /* Number of times logical_decoding_work_mem has been reached */
+ int64 memoryLimitHits;
};
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..8ce27032b87 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2140,9 +2140,10 @@ pg_stat_replication_slots| SELECT s.slot_name,
s.stream_bytes,
s.total_txns,
s.total_bytes,
+ s.memory_limit_hits,
s.stats_reset
FROM pg_replication_slots r,
- LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, total_txns, total_bytes, stats_reset)
+ LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, total_txns, total_bytes, memory_limit_hits, stats_reset)
WHERE (r.datoid IS NOT NULL);
pg_stat_slru| SELECT name,
blks_zeroed,
--
2.34.1
On Mon, Sep 22, 2025 at 1:41 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
Since other statistics counter names are camel cases I think it's
better to follow that for the new counter.Makes sense, done with memoryLimitHits in v2 attached (that's the only change
as compared with v1).
The memory_limit_hits doesn't go well with the other names in the
view. Can we consider memory_exceeded_count? I find
memory_exceeded_count (or memory_exceeds_count) more clear and
matching with the existing counters. Also, how about keeping it
immediately after slot_name in the view? Keeping it in the end after
total_bytes seems out of place.
--
With Regards,
Amit Kapila.
On Mon, Sep 22, 2025 at 4:35 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Mon, Sep 22, 2025 at 1:41 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Since other statistics counter names are camel cases I think it's
better to follow that for the new counter.Makes sense, done with memoryLimitHits in v2 attached (that's the only change
as compared with v1).The memory_limit_hits doesn't go well with the other names in the
view. Can we consider memory_exceeded_count? I find
memory_exceeded_count (or memory_exceeds_count) more clear and
matching with the existing counters. Also, how about keeping it
immediately after slot_name in the view? Keeping it in the end after
total_bytes seems out of place.
Since fields like spill_txns, spill_bytes, and stream_txns also talk
about exceeding 'logical_decoding_work_mem', my preference would be to
place this new field immediately after these spill and stream fields
(and before total_bytes). If not this, then as Amit suggested,
immediately before all these fields.
Other options for name could be 'mem_limit_exceeded_count' or
'mem_limit_hit_count'
thanks
Shveta
On Mon, Sep 22, 2025 at 05:21:35PM +0530, shveta malik wrote:
On Mon, Sep 22, 2025 at 4:35 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Mon, Sep 22, 2025 at 1:41 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Since other statistics counter names are camel cases I think it's
better to follow that for the new counter.Makes sense, done with memoryLimitHits in v2 attached (that's the only change
as compared with v1).The memory_limit_hits doesn't go well with the other names in the
view. Can we consider memory_exceeded_count? I find
memory_exceeded_count (or memory_exceeds_count) more clear and
matching with the existing counters. Also, how about keeping it
immediately after slot_name in the view? Keeping it in the end after
total_bytes seems out of place.Since fields like spill_txns, spill_bytes, and stream_txns also talk
about exceeding 'logical_decoding_work_mem', my preference would be to
place this new field immediately after these spill and stream fields
(and before total_bytes). If not this, then as Amit suggested,
immediately before all these fields.
Other options for name could be 'mem_limit_exceeded_count' or
'mem_limit_hit_count'
Thank you, Shveta and Amit, for looking at it. Since we already use txns as
abbreviation for transactions then I think it's ok to use "mem". Then I'm using
a mix of your proposals with "mem_exceeded_count" in v3 attached. Regarding the
field position, I like Shveta's proposal and did it that way.
However, technically speaking, "exceeded" is not the perfect wording since
the code was doing (and is still doing something similar to):
if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
- rb->size < logical_decoding_work_mem * (Size) 1024)
+ !memory_limit_reached)
return;as the comment describes correctly using "reached":
/*
* Check whether the logical_decoding_work_mem limit was reached, and if yes
* pick the largest (sub)transaction at-a-time to evict and spill its changes to
* disk or send to the output plugin until we reach under the memory limit.
So I think that "reached" or "hit" would be better wording. However, the
documentation for spill_txns, stream_txns already use "exceeded" (and not "reached")
so I went with "exceeded" for consistency. I think that's fine, if not we may want
to use "reached" for those 3 stats descriptions.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v3-0001-Add-mem_exceeded_count-to-pg_stat_replication_slo.patchtext/x-diff; charset=us-asciiDownload
From eb65ef2011542d15b8575a4cc0c2afacaa64cc36 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Tue, 26 Aug 2025 13:08:13 +0000
Subject: [PATCH v3] Add mem_exceeded_count to pg_stat_replication_slots
It's currently not always possible to determine how many times the
logical_decoding_work_mem has been reached.
So adding a new counter, mem_exceeded_count to report the number of times the
logical_decoding_work_mem has been reached while decoding.
With such a counter one could get a ratio like total_txns/mem_exceeded_count.
That could help to see if reaching logical_decoding_work_mem is rare or
frequent enough. If frequent, then maybe there is a need to adjust
logical_decoding_work_mem.
XXXX: Bump catversion.
---
contrib/test_decoding/expected/stats.out | 68 +++++++++----------
contrib/test_decoding/sql/stats.sql | 10 +--
doc/src/sgml/monitoring.sgml | 11 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/replication/logical/logical.c | 5 +-
.../replication/logical/reorderbuffer.c | 7 +-
src/backend/utils/activity/pgstat_replslot.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 19 +++---
src/include/catalog/pg_proc.dat | 6 +-
src/include/pgstat.h | 1 +
src/include/replication/reorderbuffer.h | 3 +
src/test/regress/expected/rules.out | 3 +-
12 files changed, 82 insertions(+), 53 deletions(-)
50.0% contrib/test_decoding/expected/
13.7% contrib/test_decoding/sql/
5.4% doc/src/sgml/
6.0% src/backend/replication/logical/
14.2% src/backend/utils/adt/
4.8% src/include/catalog/
5.4% src/
diff --git a/contrib/test_decoding/expected/stats.out b/contrib/test_decoding/expected/stats.out
index de6dc416130..0f18eea8c68 100644
--- a/contrib/test_decoding/expected/stats.out
+++ b/contrib/test_decoding/expected/stats.out
@@ -37,12 +37,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | t | t
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | t | t | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
RESET logical_decoding_work_mem;
@@ -53,12 +53,12 @@ SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
-- reset stats for all slots
@@ -68,27 +68,27 @@ SELECT pg_stat_reset_replication_slot(NULL);
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | f | f
- regression_slot_stats3 | t | t | f | f
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | f | f | t
+ regression_slot_stats3 | t | t | f | f | t
(3 rows)
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | mem_exceeded_count | total_txns | total_bytes | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+--------------------+------------+-------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
SELECT pg_stat_reset_replication_slot('do-not-exist');
ERROR: replication slot "do-not-exist" does not exist
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | mem_exceeded_count | total_txns | total_bytes | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+--------------------+------------+-------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
-- spilling the xact
@@ -110,12 +110,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
- slot_name | spill_txns | spill_count
-------------------------+------------+-------------
- regression_slot_stats1 | t | t
- regression_slot_stats2 | f | f
- regression_slot_stats3 | f | f
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, mem_exceeded_count > 0 AS mem_exceeded_count FROM pg_stat_replication_slots;
+ slot_name | spill_txns | spill_count | mem_exceeded_count
+------------------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | t
+ regression_slot_stats2 | f | f | f
+ regression_slot_stats3 | f | f | f
(3 rows)
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
@@ -165,10 +165,10 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
- slot_name | spill_txns | spill_count
----------------------------------+------------+-------------
- regression_slot_stats4_twophase | 0 | 0
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+ slot_name | spill_txns | spill_count | mem_exceeded_count
+---------------------------------+------------+-------------+--------------------
+ regression_slot_stats4_twophase | 0 | 0 | 0
(1 row)
DROP TABLE stats_test;
diff --git a/contrib/test_decoding/sql/stats.sql b/contrib/test_decoding/sql/stats.sql
index a022fe1bf07..9964a8efb87 100644
--- a/contrib/test_decoding/sql/stats.sql
+++ b/contrib/test_decoding/sql/stats.sql
@@ -15,16 +15,16 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats1', NULL,
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats2', NULL, NULL, 'skip-empty-xacts', '1');
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats3', NULL, NULL, 'skip-empty-xacts', '1');
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
RESET logical_decoding_work_mem;
-- reset stats for one slot, others should be unaffected
SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
-- reset stats for all slots
SELECT pg_stat_reset_replication_slot(NULL);
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
@@ -41,7 +41,7 @@ SELECT count(*) FROM pg_logical_slot_peek_changes('regression_slot_stats1', NULL
-- background transaction (say by autovacuum) happens in parallel to the main
-- transaction.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, mem_exceeded_count > 0 AS mem_exceeded_count FROM pg_stat_replication_slots;
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
-- https://postgr.es/m/20210317230447.c7uc4g3vbs4wi32i%40alap3.anarazel.de
@@ -65,7 +65,7 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats4_twophas
-- Verify that the decoding doesn't spill already-aborted transaction's changes.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
DROP TABLE stats_test;
SELECT pg_drop_replication_slot('regression_slot_stats1'),
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 3f4a27a736e..6d8060bc5f4 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1620,6 +1620,17 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>mem_exceeded_count</structfield><type>bigint</type>
+ </para>
+ <para>
+ Number of times <literal>logical_decoding_work_mem</literal> has been
+ exceeded while decoding changes from WAL for this slot.
+ </para>
+ </entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>total_txns</structfield> <type>bigint</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index c77fa0234bb..b18e7c42d17 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1059,6 +1059,7 @@ CREATE VIEW pg_stat_replication_slots AS
s.stream_txns,
s.stream_count,
s.stream_bytes,
+ s.mem_exceeded_count,
s.total_txns,
s.total_bytes,
s.stats_reset
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c68c0481f42..a20f0ff56e1 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1958,7 +1958,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
if (rb->spillBytes <= 0 && rb->streamBytes <= 0 && rb->totalBytes <= 0)
return;
- elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
+ elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
rb,
rb->spillTxns,
rb->spillCount,
@@ -1966,6 +1966,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamTxns,
rb->streamCount,
rb->streamBytes,
+ rb->memExceededCount,
rb->totalTxns,
rb->totalBytes);
@@ -1975,6 +1976,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
repSlotStat.stream_txns = rb->streamTxns;
repSlotStat.stream_count = rb->streamCount;
repSlotStat.stream_bytes = rb->streamBytes;
+ repSlotStat.mem_exceeded_count = rb->memExceededCount;
repSlotStat.total_txns = rb->totalTxns;
repSlotStat.total_bytes = rb->totalBytes;
@@ -1986,6 +1988,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamTxns = 0;
rb->streamCount = 0;
rb->streamBytes = 0;
+ rb->memExceededCount = 0;
rb->totalTxns = 0;
rb->totalBytes = 0;
}
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 4736f993c37..c24c2ffc8b3 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -390,6 +390,7 @@ ReorderBufferAllocate(void)
buffer->streamTxns = 0;
buffer->streamCount = 0;
buffer->streamBytes = 0;
+ buffer->memExceededCount = 0;
buffer->totalTxns = 0;
buffer->totalBytes = 0;
@@ -3898,13 +3899,17 @@ static void
ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
{
ReorderBufferTXN *txn;
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024);
+
+ if (memory_limit_reached)
+ rb->memExceededCount += 1;
/*
* Bail out if debug_logical_replication_streaming is buffered and we
* haven't exceeded the memory limit.
*/
if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
- rb->size < logical_decoding_work_mem * (Size) 1024)
+ !memory_limit_reached)
return;
/*
diff --git a/src/backend/utils/activity/pgstat_replslot.c b/src/backend/utils/activity/pgstat_replslot.c
index ccfb11c49bf..d210c261ac6 100644
--- a/src/backend/utils/activity/pgstat_replslot.c
+++ b/src/backend/utils/activity/pgstat_replslot.c
@@ -94,6 +94,7 @@ pgstat_report_replslot(ReplicationSlot *slot, const PgStat_StatReplSlotEntry *re
REPLSLOT_ACC(stream_txns);
REPLSLOT_ACC(stream_count);
REPLSLOT_ACC(stream_bytes);
+ REPLSLOT_ACC(mem_exceeded_count);
REPLSLOT_ACC(total_txns);
REPLSLOT_ACC(total_bytes);
#undef REPLSLOT_ACC
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..e64dd5e043e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2100,7 +2100,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
Datum
pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_REPLICATION_SLOT_COLS 10
+#define PG_STAT_GET_REPLICATION_SLOT_COLS 11
text *slotname_text = PG_GETARG_TEXT_P(0);
NameData slotname;
TupleDesc tupdesc;
@@ -2125,11 +2125,13 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stream_bytes",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 8, "total_txns",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "mem_exceeded_count",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_bytes",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_txns",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 10, "stats_reset",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 10, "total_bytes",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 11, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
@@ -2152,13 +2154,14 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
values[4] = Int64GetDatum(slotent->stream_txns);
values[5] = Int64GetDatum(slotent->stream_count);
values[6] = Int64GetDatum(slotent->stream_bytes);
- values[7] = Int64GetDatum(slotent->total_txns);
- values[8] = Int64GetDatum(slotent->total_bytes);
+ values[7] = Int64GetDatum(slotent->mem_exceeded_count);
+ values[8] = Int64GetDatum(slotent->total_txns);
+ values[9] = Int64GetDatum(slotent->total_bytes);
if (slotent->stat_reset_timestamp == 0)
- nulls[9] = true;
+ nulls[10] = true;
else
- values[9] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
+ values[10] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 01eba3b5a19..4ea72c85240 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5687,9 +5687,9 @@
{ oid => '6169', descr => 'statistics: information about replication slot',
proname => 'pg_stat_get_replication_slot', provolatile => 's',
proparallel => 'r', prorettype => 'record', proargtypes => 'text',
- proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
+ proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,mem_exceeded_count,total_txns,total_bytes,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
{ oid => '6230', descr => 'statistics: check if a stats object exists',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f402b17295c..2a38b431e8e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -393,6 +393,7 @@ typedef struct PgStat_StatReplSlotEntry
PgStat_Counter stream_txns;
PgStat_Counter stream_count;
PgStat_Counter stream_bytes;
+ PgStat_Counter mem_exceeded_count;
PgStat_Counter total_txns;
PgStat_Counter total_bytes;
TimestampTz stat_reset_timestamp;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index fa0745552f8..c8edbb7f0e9 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -690,6 +690,9 @@ struct ReorderBuffer
int64 streamCount; /* streaming invocation counter */
int64 streamBytes; /* amount of data decoded */
+ /* Number of times logical_decoding_work_mem has been reached */
+ int64 memExceededCount;
+
/*
* Statistics about all the transactions sent to the decoding output
* plugin
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..f4f3a2a3018 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2138,11 +2138,12 @@ pg_stat_replication_slots| SELECT s.slot_name,
s.stream_txns,
s.stream_count,
s.stream_bytes,
+ s.mem_exceeded_count,
s.total_txns,
s.total_bytes,
s.stats_reset
FROM pg_replication_slots r,
- LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, total_txns, total_bytes, stats_reset)
+ LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, mem_exceeded_count, total_txns, total_bytes, stats_reset)
WHERE (r.datoid IS NOT NULL);
pg_stat_slru| SELECT name,
blks_zeroed,
--
2.34.1
On Tue, Sep 23, 2025 at 1:52 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
On Mon, Sep 22, 2025 at 05:21:35PM +0530, shveta malik wrote:
On Mon, Sep 22, 2025 at 4:35 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Mon, Sep 22, 2025 at 1:41 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Since other statistics counter names are camel cases I think it's
better to follow that for the new counter.Makes sense, done with memoryLimitHits in v2 attached (that's the only change
as compared with v1).The memory_limit_hits doesn't go well with the other names in the
view. Can we consider memory_exceeded_count? I find
memory_exceeded_count (or memory_exceeds_count) more clear and
matching with the existing counters. Also, how about keeping it
immediately after slot_name in the view? Keeping it in the end after
total_bytes seems out of place.Since fields like spill_txns, spill_bytes, and stream_txns also talk
about exceeding 'logical_decoding_work_mem', my preference would be to
place this new field immediately after these spill and stream fields
(and before total_bytes). If not this, then as Amit suggested,
immediately before all these fields.
Other options for name could be 'mem_limit_exceeded_count' or
'mem_limit_hit_count'Thank you, Shveta and Amit, for looking at it. Since we already use txns as
abbreviation for transactions then I think it's ok to use "mem". Then I'm using
a mix of your proposals with "mem_exceeded_count" in v3 attached. Regarding the
field position, I like Shveta's proposal and did it that way.However, technically speaking, "exceeded" is not the perfect wording since
the code was doing (and is still doing something similar to):if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED && - rb->size < logical_decoding_work_mem * (Size) 1024) + !memory_limit_reached) return;as the comment describes correctly using "reached":
/*
* Check whether the logical_decoding_work_mem limit was reached, and if yes
* pick the largest (sub)transaction at-a-time to evict and spill its changes to
* disk or send to the output plugin until we reach under the memory limit.So I think that "reached" or "hit" would be better wording. However, the
documentation for spill_txns, stream_txns already use "exceeded" (and not "reached")
so I went with "exceeded" for consistency. I think that's fine, if not we may want
to use "reached" for those 3 stats descriptions.
I find "exceeded" is fine as the documentation for logical decoding
also uses it[1]https://www.postgresql.org/docs/devel/logicaldecoding-streaming.html:
Similar to spill-to-disk behavior, streaming is triggered when the
total amount of changes decoded from the WAL (for all in-progress
transactions) exceeds the limit defined by logical_decoding_work_mem
setting.
One comment for the v3 patch:
+ <para>
+ Number of times <literal>logical_decoding_work_mem</literal> has been
+ exceeded while decoding changes from WAL for this slot.
+ </para>How about rewording it to like:
Number of times the memory used by logical decoding has exceeded
logical_decoding_work_mem.
Regards,
[1]: https://www.postgresql.org/docs/devel/logicaldecoding-streaming.html
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Hi,
On Tue, Sep 23, 2025 at 11:39:22AM -0700, Masahiko Sawada wrote:
On Tue, Sep 23, 2025 at 1:52 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:However, technically speaking, "exceeded" is not the perfect wording since
the code was doing (and is still doing something similar to):if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED && - rb->size < logical_decoding_work_mem * (Size) 1024) + !memory_limit_reached) return;as the comment describes correctly using "reached":
/*
* Check whether the logical_decoding_work_mem limit was reached, and if yes
* pick the largest (sub)transaction at-a-time to evict and spill its changes to
* disk or send to the output plugin until we reach under the memory limit.So I think that "reached" or "hit" would be better wording. However, the
documentation for spill_txns, stream_txns already use "exceeded" (and not "reached")
so I went with "exceeded" for consistency. I think that's fine, if not we may want
to use "reached" for those 3 stats descriptions.I find "exceeded" is fine as the documentation for logical decoding
also uses it[1]:Similar to spill-to-disk behavior, streaming is triggered when the
total amount of changes decoded from the WAL (for all in-progress
transactions) exceeds the limit defined by logical_decoding_work_mem
setting.
Yes it also uses "exceeds" but I think it's not 100% accurate. It would be
if, in ReorderBufferCheckMemoryLimit, we were using "<=" instead of "<" in:
if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
rb->size < logical_decoding_work_mem * (Size) 1024)
I think an accurate wording would be "reaches or exceeds" in all those places,
but just using "exceeds" looks good enough.
One comment for the v3 patch:
+ <para> + Number of times <literal>logical_decoding_work_mem</literal> has been + exceeded while decoding changes from WAL for this slot. + </para>How about rewording it to like:
Number of times the memory used by logical decoding has exceeded
logical_decoding_work_mem.
That sounds better, thanks! Used this wording in v4 attached (that's the only
change as compared to v3).
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v4-0001-Add-mem_exceeded_count-to-pg_stat_replication_slo.patchtext/x-diff; charset=us-asciiDownload
From c225f3471b891db4cf2e37f009b381a794ee1ffb Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Tue, 26 Aug 2025 13:08:13 +0000
Subject: [PATCH v4] Add mem_exceeded_count to pg_stat_replication_slots
It's currently not always possible to determine how many times the
logical_decoding_work_mem has been reached.
So adding a new counter, mem_exceeded_count to report the number of times the
logical_decoding_work_mem has been reached while decoding.
With such a counter one could get a ratio like total_txns/mem_exceeded_count.
That could help to see if reaching logical_decoding_work_mem is rare or
frequent enough. If frequent, then maybe there is a need to adjust
logical_decoding_work_mem.
XXXX: Bump catversion.
---
contrib/test_decoding/expected/stats.out | 68 +++++++++----------
contrib/test_decoding/sql/stats.sql | 10 +--
doc/src/sgml/monitoring.sgml | 11 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/replication/logical/logical.c | 5 +-
.../replication/logical/reorderbuffer.c | 7 +-
src/backend/utils/activity/pgstat_replslot.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 19 +++---
src/include/catalog/pg_proc.dat | 6 +-
src/include/pgstat.h | 1 +
src/include/replication/reorderbuffer.h | 3 +
src/test/regress/expected/rules.out | 3 +-
12 files changed, 82 insertions(+), 53 deletions(-)
50.2% contrib/test_decoding/expected/
13.7% contrib/test_decoding/sql/
5.2% doc/src/sgml/
6.1% src/backend/replication/logical/
14.3% src/backend/utils/adt/
4.8% src/include/catalog/
5.4% src/
diff --git a/contrib/test_decoding/expected/stats.out b/contrib/test_decoding/expected/stats.out
index de6dc416130..0f18eea8c68 100644
--- a/contrib/test_decoding/expected/stats.out
+++ b/contrib/test_decoding/expected/stats.out
@@ -37,12 +37,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | t | t
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | t | t | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
RESET logical_decoding_work_mem;
@@ -53,12 +53,12 @@ SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
-- reset stats for all slots
@@ -68,27 +68,27 @@ SELECT pg_stat_reset_replication_slot(NULL);
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | f | f
- regression_slot_stats3 | t | t | f | f
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | f | f | t
+ regression_slot_stats3 | t | t | f | f | t
(3 rows)
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | mem_exceeded_count | total_txns | total_bytes | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+--------------------+------------+-------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
SELECT pg_stat_reset_replication_slot('do-not-exist');
ERROR: replication slot "do-not-exist" does not exist
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | mem_exceeded_count | total_txns | total_bytes | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+--------------------+------------+-------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
-- spilling the xact
@@ -110,12 +110,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
- slot_name | spill_txns | spill_count
-------------------------+------------+-------------
- regression_slot_stats1 | t | t
- regression_slot_stats2 | f | f
- regression_slot_stats3 | f | f
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, mem_exceeded_count > 0 AS mem_exceeded_count FROM pg_stat_replication_slots;
+ slot_name | spill_txns | spill_count | mem_exceeded_count
+------------------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | t
+ regression_slot_stats2 | f | f | f
+ regression_slot_stats3 | f | f | f
(3 rows)
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
@@ -165,10 +165,10 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
- slot_name | spill_txns | spill_count
----------------------------------+------------+-------------
- regression_slot_stats4_twophase | 0 | 0
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+ slot_name | spill_txns | spill_count | mem_exceeded_count
+---------------------------------+------------+-------------+--------------------
+ regression_slot_stats4_twophase | 0 | 0 | 0
(1 row)
DROP TABLE stats_test;
diff --git a/contrib/test_decoding/sql/stats.sql b/contrib/test_decoding/sql/stats.sql
index a022fe1bf07..9964a8efb87 100644
--- a/contrib/test_decoding/sql/stats.sql
+++ b/contrib/test_decoding/sql/stats.sql
@@ -15,16 +15,16 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats1', NULL,
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats2', NULL, NULL, 'skip-empty-xacts', '1');
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats3', NULL, NULL, 'skip-empty-xacts', '1');
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
RESET logical_decoding_work_mem;
-- reset stats for one slot, others should be unaffected
SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
-- reset stats for all slots
SELECT pg_stat_reset_replication_slot(NULL);
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
@@ -41,7 +41,7 @@ SELECT count(*) FROM pg_logical_slot_peek_changes('regression_slot_stats1', NULL
-- background transaction (say by autovacuum) happens in parallel to the main
-- transaction.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, mem_exceeded_count > 0 AS mem_exceeded_count FROM pg_stat_replication_slots;
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
-- https://postgr.es/m/20210317230447.c7uc4g3vbs4wi32i%40alap3.anarazel.de
@@ -65,7 +65,7 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats4_twophas
-- Verify that the decoding doesn't spill already-aborted transaction's changes.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
DROP TABLE stats_test;
SELECT pg_drop_replication_slot('regression_slot_stats1'),
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 3f4a27a736e..0141c00e666 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1620,6 +1620,17 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>mem_exceeded_count</structfield><type>bigint</type>
+ </para>
+ <para>
+ Number of times the memory used by logical decoding has exceeded
+ <literal>logical_decoding_work_mem</literal>.
+ </para>
+ </entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>total_txns</structfield> <type>bigint</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index c77fa0234bb..b18e7c42d17 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1059,6 +1059,7 @@ CREATE VIEW pg_stat_replication_slots AS
s.stream_txns,
s.stream_count,
s.stream_bytes,
+ s.mem_exceeded_count,
s.total_txns,
s.total_bytes,
s.stats_reset
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c68c0481f42..a20f0ff56e1 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1958,7 +1958,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
if (rb->spillBytes <= 0 && rb->streamBytes <= 0 && rb->totalBytes <= 0)
return;
- elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
+ elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
rb,
rb->spillTxns,
rb->spillCount,
@@ -1966,6 +1966,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamTxns,
rb->streamCount,
rb->streamBytes,
+ rb->memExceededCount,
rb->totalTxns,
rb->totalBytes);
@@ -1975,6 +1976,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
repSlotStat.stream_txns = rb->streamTxns;
repSlotStat.stream_count = rb->streamCount;
repSlotStat.stream_bytes = rb->streamBytes;
+ repSlotStat.mem_exceeded_count = rb->memExceededCount;
repSlotStat.total_txns = rb->totalTxns;
repSlotStat.total_bytes = rb->totalBytes;
@@ -1986,6 +1988,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamTxns = 0;
rb->streamCount = 0;
rb->streamBytes = 0;
+ rb->memExceededCount = 0;
rb->totalTxns = 0;
rb->totalBytes = 0;
}
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 4736f993c37..c24c2ffc8b3 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -390,6 +390,7 @@ ReorderBufferAllocate(void)
buffer->streamTxns = 0;
buffer->streamCount = 0;
buffer->streamBytes = 0;
+ buffer->memExceededCount = 0;
buffer->totalTxns = 0;
buffer->totalBytes = 0;
@@ -3898,13 +3899,17 @@ static void
ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
{
ReorderBufferTXN *txn;
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024);
+
+ if (memory_limit_reached)
+ rb->memExceededCount += 1;
/*
* Bail out if debug_logical_replication_streaming is buffered and we
* haven't exceeded the memory limit.
*/
if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
- rb->size < logical_decoding_work_mem * (Size) 1024)
+ !memory_limit_reached)
return;
/*
diff --git a/src/backend/utils/activity/pgstat_replslot.c b/src/backend/utils/activity/pgstat_replslot.c
index ccfb11c49bf..d210c261ac6 100644
--- a/src/backend/utils/activity/pgstat_replslot.c
+++ b/src/backend/utils/activity/pgstat_replslot.c
@@ -94,6 +94,7 @@ pgstat_report_replslot(ReplicationSlot *slot, const PgStat_StatReplSlotEntry *re
REPLSLOT_ACC(stream_txns);
REPLSLOT_ACC(stream_count);
REPLSLOT_ACC(stream_bytes);
+ REPLSLOT_ACC(mem_exceeded_count);
REPLSLOT_ACC(total_txns);
REPLSLOT_ACC(total_bytes);
#undef REPLSLOT_ACC
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..e64dd5e043e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2100,7 +2100,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
Datum
pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_REPLICATION_SLOT_COLS 10
+#define PG_STAT_GET_REPLICATION_SLOT_COLS 11
text *slotname_text = PG_GETARG_TEXT_P(0);
NameData slotname;
TupleDesc tupdesc;
@@ -2125,11 +2125,13 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stream_bytes",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 8, "total_txns",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "mem_exceeded_count",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_bytes",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_txns",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 10, "stats_reset",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 10, "total_bytes",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 11, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
@@ -2152,13 +2154,14 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
values[4] = Int64GetDatum(slotent->stream_txns);
values[5] = Int64GetDatum(slotent->stream_count);
values[6] = Int64GetDatum(slotent->stream_bytes);
- values[7] = Int64GetDatum(slotent->total_txns);
- values[8] = Int64GetDatum(slotent->total_bytes);
+ values[7] = Int64GetDatum(slotent->mem_exceeded_count);
+ values[8] = Int64GetDatum(slotent->total_txns);
+ values[9] = Int64GetDatum(slotent->total_bytes);
if (slotent->stat_reset_timestamp == 0)
- nulls[9] = true;
+ nulls[10] = true;
else
- values[9] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
+ values[10] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 01eba3b5a19..4ea72c85240 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5687,9 +5687,9 @@
{ oid => '6169', descr => 'statistics: information about replication slot',
proname => 'pg_stat_get_replication_slot', provolatile => 's',
proparallel => 'r', prorettype => 'record', proargtypes => 'text',
- proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
+ proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,mem_exceeded_count,total_txns,total_bytes,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
{ oid => '6230', descr => 'statistics: check if a stats object exists',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f402b17295c..2a38b431e8e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -393,6 +393,7 @@ typedef struct PgStat_StatReplSlotEntry
PgStat_Counter stream_txns;
PgStat_Counter stream_count;
PgStat_Counter stream_bytes;
+ PgStat_Counter mem_exceeded_count;
PgStat_Counter total_txns;
PgStat_Counter total_bytes;
TimestampTz stat_reset_timestamp;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index fa0745552f8..c8edbb7f0e9 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -690,6 +690,9 @@ struct ReorderBuffer
int64 streamCount; /* streaming invocation counter */
int64 streamBytes; /* amount of data decoded */
+ /* Number of times logical_decoding_work_mem has been reached */
+ int64 memExceededCount;
+
/*
* Statistics about all the transactions sent to the decoding output
* plugin
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..f4f3a2a3018 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2138,11 +2138,12 @@ pg_stat_replication_slots| SELECT s.slot_name,
s.stream_txns,
s.stream_count,
s.stream_bytes,
+ s.mem_exceeded_count,
s.total_txns,
s.total_bytes,
s.stats_reset
FROM pg_replication_slots r,
- LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, total_txns, total_bytes, stats_reset)
+ LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, mem_exceeded_count, total_txns, total_bytes, stats_reset)
WHERE (r.datoid IS NOT NULL);
pg_stat_slru| SELECT name,
blks_zeroed,
--
2.34.1
On Tue, Sep 23, 2025 at 11:31 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
Hi,
On Tue, Sep 23, 2025 at 11:39:22AM -0700, Masahiko Sawada wrote:
On Tue, Sep 23, 2025 at 1:52 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:However, technically speaking, "exceeded" is not the perfect wording since
the code was doing (and is still doing something similar to):if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED && - rb->size < logical_decoding_work_mem * (Size) 1024) + !memory_limit_reached) return;as the comment describes correctly using "reached":
/*
* Check whether the logical_decoding_work_mem limit was reached, and if yes
* pick the largest (sub)transaction at-a-time to evict and spill its changes to
* disk or send to the output plugin until we reach under the memory limit.So I think that "reached" or "hit" would be better wording. However, the
documentation for spill_txns, stream_txns already use "exceeded" (and not "reached")
so I went with "exceeded" for consistency. I think that's fine, if not we may want
to use "reached" for those 3 stats descriptions.I find "exceeded" is fine as the documentation for logical decoding
also uses it[1]:Similar to spill-to-disk behavior, streaming is triggered when the
total amount of changes decoded from the WAL (for all in-progress
transactions) exceeds the limit defined by logical_decoding_work_mem
setting.Yes it also uses "exceeds" but I think it's not 100% accurate. It would be
if, in ReorderBufferCheckMemoryLimit, we were using "<=" instead of "<" in:if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
rb->size < logical_decoding_work_mem * (Size) 1024)I think an accurate wording would be "reaches or exceeds" in all those places,
but just using "exceeds" looks good enough.One comment for the v3 patch:
+ <para> + Number of times <literal>logical_decoding_work_mem</literal> has been + exceeded while decoding changes from WAL for this slot. + </para>How about rewording it to like:
Number of times the memory used by logical decoding has exceeded
logical_decoding_work_mem.That sounds better, thanks! Used this wording in v4 attached (that's the only
change as compared to v3).
Thank you for updating the patch! Here are some comments:
---
+ bool memory_limit_reached = (rb->size >=
logical_decoding_work_mem * (Size) 1024);
+
+ if (memory_limit_reached)
+ rb->memExceededCount += 1;Do we want to use 'exceeded' for the variable too for better consistency?
---
One thing I want to clarify is that even if the memory usage exceeds
the logical_decoding_work_mem it doesn't necessarily mean we serialize
or stream transactions because of
ReorderBufferCheckAndTruncateAbortedTXN(). For example, in a situation
where many large already-aborted transactions are truncated by
transactionsReorderBufferCheckAndTruncateAbortedTXN(), users would see
a high number in mem_exceeded_count column but it might not actually
require any adjustment for logical_decoding_work_mem. One idea is to
increment that counter if exceeding memory usage is caused to
serialize or stream any transactions. On the other hand, it might make
sense and be straightforward too to show a pure statistic that the
memory usage exceeded the logical_decoding_work_mem. What do you
think?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Hi,
On Wed, Sep 24, 2025 at 10:11:20AM -0700, Masahiko Sawada wrote:
On Tue, Sep 23, 2025 at 11:31 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Thank you for updating the patch! Here are some comments:
--- + bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024); + + if (memory_limit_reached) + rb->memExceededCount += 1;Do we want to use 'exceeded' for the variable too for better consistency?
I thought about it, but since we use ">=" I think that "reached" is more
accurate. So I went for "reached" for this one and "exceeded" for "user facing"
ones. That said I don't have a strong opinion about it, and I'd be ok to use
"exceeded" if you feel strong about it.
---
One thing I want to clarify is that even if the memory usage exceeds
the logical_decoding_work_mem it doesn't necessarily mean we serialize
or stream transactions because of
ReorderBufferCheckAndTruncateAbortedTXN().
Right.
For example, in a situation
where many large already-aborted transactions are truncated by
transactionsReorderBufferCheckAndTruncateAbortedTXN(), users would see
a high number in mem_exceeded_count column but it might not actually
require any adjustment for logical_decoding_work_mem.
Yes, but in that case mem_exceeded_count would be high compared to spill_txns,
stream_txns, no?
One idea is to
increment that counter if exceeding memory usage is caused to
serialize or stream any transactions. On the other hand, it might make
sense and be straightforward too to show a pure statistic that the
memory usage exceeded the logical_decoding_work_mem. What do you
think?
The new counter, as it is proposed, helps to see if the workload hits the
logical_decoding_work_mem frequently or not. I think it's valuable information
to have on its own.
Now to check if logical_decoding_work_mem needs adjustment, one could compare
mem_exceeded_count with the existing spill_txns and stream_txns.
For example, If I abort 20 transactions that exceeded logical_decoding_work_mem
, I'd get:
postgres=# select spill_txns,stream_txns,mem_exceeded_count from pg_stat_replication_slots ;
spill_txns | stream_txns | mem_exceeded_count
------------+-------------+--------------------
0 | 0 | 20
(1 row)
That way I could figure out that mem_exceeded_count has been reached for
aborted transactions.
OTOH, If one see spill_txns + stream_txns close to mem_exceeded_count, like:
postgres=# select spill_txns,stream_txns,mem_exceeded_count from pg_stat_replication_slots ;
spill_txns | stream_txns | mem_exceeded_count
------------+-------------+--------------------
38 | 20 | 58
(1 row)
That probably means that mem_exceeded_count would need to be increased.
What do you think?
BTW, while doing some tests for the above examples, I noticed that the patch
was missing a check on memExceededCount in UpdateDecodingStats() (that produced
mem_exceeded_count being 0 for one of the new test in test_decoding): Fixed in
v5 attached.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachments:
v5-0001-Add-mem_exceeded_count-to-pg_stat_replication_slo.patchtext/x-diff; charset=us-asciiDownload
From 2abb6f4eac0222bd8823935d69c8353b89281642 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Tue, 26 Aug 2025 13:08:13 +0000
Subject: [PATCH v5] Add mem_exceeded_count to pg_stat_replication_slots
It's currently not always possible to determine how many times the
logical_decoding_work_mem has been reached.
So adding a new counter, mem_exceeded_count to report the number of times the
logical_decoding_work_mem has been reached while decoding.
With such a counter one could get a ratio like total_txns/mem_exceeded_count.
That could help to see if reaching logical_decoding_work_mem is rare or
frequent enough. If frequent, then maybe there is a need to adjust
logical_decoding_work_mem.
XXXX: Bump catversion.
---
contrib/test_decoding/expected/stats.out | 68 +++++++++----------
contrib/test_decoding/sql/stats.sql | 10 +--
doc/src/sgml/monitoring.sgml | 11 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/replication/logical/logical.c | 8 ++-
.../replication/logical/reorderbuffer.c | 7 +-
src/backend/utils/activity/pgstat_replslot.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 19 +++---
src/include/catalog/pg_proc.dat | 6 +-
src/include/pgstat.h | 1 +
src/include/replication/reorderbuffer.h | 3 +
src/test/regress/expected/rules.out | 3 +-
12 files changed, 84 insertions(+), 54 deletions(-)
49.8% contrib/test_decoding/expected/
13.6% contrib/test_decoding/sql/
5.1% doc/src/sgml/
6.7% src/backend/replication/logical/
14.1% src/backend/utils/adt/
4.8% src/include/catalog/
5.4% src/
diff --git a/contrib/test_decoding/expected/stats.out b/contrib/test_decoding/expected/stats.out
index de6dc416130..72fbb270334 100644
--- a/contrib/test_decoding/expected/stats.out
+++ b/contrib/test_decoding/expected/stats.out
@@ -37,12 +37,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | t | t
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | t | t | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
RESET logical_decoding_work_mem;
@@ -53,12 +53,12 @@ SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
-- reset stats for all slots
@@ -68,27 +68,27 @@ SELECT pg_stat_reset_replication_slot(NULL);
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | f | f
- regression_slot_stats3 | t | t | f | f
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | f | f | t
+ regression_slot_stats3 | t | t | f | f | t
(3 rows)
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | mem_exceeded_count | total_txns | total_bytes | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+--------------------+------------+-------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
SELECT pg_stat_reset_replication_slot('do-not-exist');
ERROR: replication slot "do-not-exist" does not exist
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | mem_exceeded_count | total_txns | total_bytes | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+--------------------+------------+-------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
-- spilling the xact
@@ -110,12 +110,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
- slot_name | spill_txns | spill_count
-------------------------+------------+-------------
- regression_slot_stats1 | t | t
- regression_slot_stats2 | f | f
- regression_slot_stats3 | f | f
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, mem_exceeded_count > 0 AS mem_exceeded_count FROM pg_stat_replication_slots;
+ slot_name | spill_txns | spill_count | mem_exceeded_count
+------------------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | t
+ regression_slot_stats2 | f | f | f
+ regression_slot_stats3 | f | f | f
(3 rows)
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
@@ -165,10 +165,10 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
- slot_name | spill_txns | spill_count
----------------------------------+------------+-------------
- regression_slot_stats4_twophase | 0 | 0
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+ slot_name | spill_txns | spill_count | mem_exceeded_count
+---------------------------------+------------+-------------+--------------------
+ regression_slot_stats4_twophase | 0 | 0 | 1
(1 row)
DROP TABLE stats_test;
diff --git a/contrib/test_decoding/sql/stats.sql b/contrib/test_decoding/sql/stats.sql
index a022fe1bf07..9964a8efb87 100644
--- a/contrib/test_decoding/sql/stats.sql
+++ b/contrib/test_decoding/sql/stats.sql
@@ -15,16 +15,16 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats1', NULL,
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats2', NULL, NULL, 'skip-empty-xacts', '1');
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats3', NULL, NULL, 'skip-empty-xacts', '1');
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
RESET logical_decoding_work_mem;
-- reset stats for one slot, others should be unaffected
SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
-- reset stats for all slots
SELECT pg_stat_reset_replication_slot(NULL);
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
@@ -41,7 +41,7 @@ SELECT count(*) FROM pg_logical_slot_peek_changes('regression_slot_stats1', NULL
-- background transaction (say by autovacuum) happens in parallel to the main
-- transaction.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, mem_exceeded_count > 0 AS mem_exceeded_count FROM pg_stat_replication_slots;
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
-- https://postgr.es/m/20210317230447.c7uc4g3vbs4wi32i%40alap3.anarazel.de
@@ -65,7 +65,7 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats4_twophas
-- Verify that the decoding doesn't spill already-aborted transaction's changes.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
DROP TABLE stats_test;
SELECT pg_drop_replication_slot('regression_slot_stats1'),
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 3f4a27a736e..0141c00e666 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1620,6 +1620,17 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>mem_exceeded_count</structfield><type>bigint</type>
+ </para>
+ <para>
+ Number of times the memory used by logical decoding has exceeded
+ <literal>logical_decoding_work_mem</literal>.
+ </para>
+ </entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>total_txns</structfield> <type>bigint</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index c77fa0234bb..b18e7c42d17 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1059,6 +1059,7 @@ CREATE VIEW pg_stat_replication_slots AS
s.stream_txns,
s.stream_count,
s.stream_bytes,
+ s.mem_exceeded_count,
s.total_txns,
s.total_bytes,
s.stats_reset
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c68c0481f42..93ed2eb368e 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1955,10 +1955,11 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
PgStat_StatReplSlotEntry repSlotStat;
/* Nothing to do if we don't have any replication stats to be sent. */
- if (rb->spillBytes <= 0 && rb->streamBytes <= 0 && rb->totalBytes <= 0)
+ if (rb->spillBytes <= 0 && rb->streamBytes <= 0 && rb->totalBytes <= 0 &&
+ rb->memExceededCount <= 0)
return;
- elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
+ elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
rb,
rb->spillTxns,
rb->spillCount,
@@ -1966,6 +1967,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamTxns,
rb->streamCount,
rb->streamBytes,
+ rb->memExceededCount,
rb->totalTxns,
rb->totalBytes);
@@ -1975,6 +1977,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
repSlotStat.stream_txns = rb->streamTxns;
repSlotStat.stream_count = rb->streamCount;
repSlotStat.stream_bytes = rb->streamBytes;
+ repSlotStat.mem_exceeded_count = rb->memExceededCount;
repSlotStat.total_txns = rb->totalTxns;
repSlotStat.total_bytes = rb->totalBytes;
@@ -1986,6 +1989,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamTxns = 0;
rb->streamCount = 0;
rb->streamBytes = 0;
+ rb->memExceededCount = 0;
rb->totalTxns = 0;
rb->totalBytes = 0;
}
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 4736f993c37..c24c2ffc8b3 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -390,6 +390,7 @@ ReorderBufferAllocate(void)
buffer->streamTxns = 0;
buffer->streamCount = 0;
buffer->streamBytes = 0;
+ buffer->memExceededCount = 0;
buffer->totalTxns = 0;
buffer->totalBytes = 0;
@@ -3898,13 +3899,17 @@ static void
ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
{
ReorderBufferTXN *txn;
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024);
+
+ if (memory_limit_reached)
+ rb->memExceededCount += 1;
/*
* Bail out if debug_logical_replication_streaming is buffered and we
* haven't exceeded the memory limit.
*/
if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
- rb->size < logical_decoding_work_mem * (Size) 1024)
+ !memory_limit_reached)
return;
/*
diff --git a/src/backend/utils/activity/pgstat_replslot.c b/src/backend/utils/activity/pgstat_replslot.c
index ccfb11c49bf..d210c261ac6 100644
--- a/src/backend/utils/activity/pgstat_replslot.c
+++ b/src/backend/utils/activity/pgstat_replslot.c
@@ -94,6 +94,7 @@ pgstat_report_replslot(ReplicationSlot *slot, const PgStat_StatReplSlotEntry *re
REPLSLOT_ACC(stream_txns);
REPLSLOT_ACC(stream_count);
REPLSLOT_ACC(stream_bytes);
+ REPLSLOT_ACC(mem_exceeded_count);
REPLSLOT_ACC(total_txns);
REPLSLOT_ACC(total_bytes);
#undef REPLSLOT_ACC
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..e64dd5e043e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2100,7 +2100,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
Datum
pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_REPLICATION_SLOT_COLS 10
+#define PG_STAT_GET_REPLICATION_SLOT_COLS 11
text *slotname_text = PG_GETARG_TEXT_P(0);
NameData slotname;
TupleDesc tupdesc;
@@ -2125,11 +2125,13 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stream_bytes",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 8, "total_txns",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "mem_exceeded_count",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_bytes",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_txns",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 10, "stats_reset",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 10, "total_bytes",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 11, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
@@ -2152,13 +2154,14 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
values[4] = Int64GetDatum(slotent->stream_txns);
values[5] = Int64GetDatum(slotent->stream_count);
values[6] = Int64GetDatum(slotent->stream_bytes);
- values[7] = Int64GetDatum(slotent->total_txns);
- values[8] = Int64GetDatum(slotent->total_bytes);
+ values[7] = Int64GetDatum(slotent->mem_exceeded_count);
+ values[8] = Int64GetDatum(slotent->total_txns);
+ values[9] = Int64GetDatum(slotent->total_bytes);
if (slotent->stat_reset_timestamp == 0)
- nulls[9] = true;
+ nulls[10] = true;
else
- values[9] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
+ values[10] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 01eba3b5a19..4ea72c85240 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5687,9 +5687,9 @@
{ oid => '6169', descr => 'statistics: information about replication slot',
proname => 'pg_stat_get_replication_slot', provolatile => 's',
proparallel => 'r', prorettype => 'record', proargtypes => 'text',
- proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
+ proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,mem_exceeded_count,total_txns,total_bytes,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
{ oid => '6230', descr => 'statistics: check if a stats object exists',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index f402b17295c..2a38b431e8e 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -393,6 +393,7 @@ typedef struct PgStat_StatReplSlotEntry
PgStat_Counter stream_txns;
PgStat_Counter stream_count;
PgStat_Counter stream_bytes;
+ PgStat_Counter mem_exceeded_count;
PgStat_Counter total_txns;
PgStat_Counter total_bytes;
TimestampTz stat_reset_timestamp;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index fa0745552f8..c8edbb7f0e9 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -690,6 +690,9 @@ struct ReorderBuffer
int64 streamCount; /* streaming invocation counter */
int64 streamBytes; /* amount of data decoded */
+ /* Number of times logical_decoding_work_mem has been reached */
+ int64 memExceededCount;
+
/*
* Statistics about all the transactions sent to the decoding output
* plugin
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..f4f3a2a3018 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2138,11 +2138,12 @@ pg_stat_replication_slots| SELECT s.slot_name,
s.stream_txns,
s.stream_count,
s.stream_bytes,
+ s.mem_exceeded_count,
s.total_txns,
s.total_bytes,
s.stats_reset
FROM pg_replication_slots r,
- LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, total_txns, total_bytes, stats_reset)
+ LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, mem_exceeded_count, total_txns, total_bytes, stats_reset)
WHERE (r.datoid IS NOT NULL);
pg_stat_slru| SELECT name,
blks_zeroed,
--
2.34.1
On Thu, Sep 25, 2025 at 3:17 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
Hi,
On Wed, Sep 24, 2025 at 10:11:20AM -0700, Masahiko Sawada wrote:
On Tue, Sep 23, 2025 at 11:31 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Thank you for updating the patch! Here are some comments:
--- + bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024); + + if (memory_limit_reached) + rb->memExceededCount += 1;Do we want to use 'exceeded' for the variable too for better consistency?
I thought about it, but since we use ">=" I think that "reached" is more
accurate. So I went for "reached" for this one and "exceeded" for "user facing"
ones. That said I don't have a strong opinion about it, and I'd be ok to use
"exceeded" if you feel strong about it.
Agreed with the current style. Thank you for the explanation.
---
One thing I want to clarify is that even if the memory usage exceeds
the logical_decoding_work_mem it doesn't necessarily mean we serialize
or stream transactions because of
ReorderBufferCheckAndTruncateAbortedTXN().Right.
For example, in a situation
where many large already-aborted transactions are truncated by
transactionsReorderBufferCheckAndTruncateAbortedTXN(), users would see
a high number in mem_exceeded_count column but it might not actually
require any adjustment for logical_decoding_work_mem.Yes, but in that case mem_exceeded_count would be high compared to spill_txns,
stream_txns, no?
Right. I think only mem_exceeded_count has a high number while
spill_txns and stream_txns have lower numbers in this case (like you
shown in your first example below).
One idea is to
increment that counter if exceeding memory usage is caused to
serialize or stream any transactions. On the other hand, it might make
sense and be straightforward too to show a pure statistic that the
memory usage exceeded the logical_decoding_work_mem. What do you
think?The new counter, as it is proposed, helps to see if the workload hits the
logical_decoding_work_mem frequently or not. I think it's valuable information
to have on its own.Now to check if logical_decoding_work_mem needs adjustment, one could compare
mem_exceeded_count with the existing spill_txns and stream_txns.For example, If I abort 20 transactions that exceeded logical_decoding_work_mem
, I'd get:postgres=# select spill_txns,stream_txns,mem_exceeded_count from pg_stat_replication_slots ;
spill_txns | stream_txns | mem_exceeded_count
------------+-------------+--------------------
0 | 0 | 20
(1 row)That way I could figure out that mem_exceeded_count has been reached for
aborted transactions.OTOH, If one see spill_txns + stream_txns close to mem_exceeded_count, like:
postgres=# select spill_txns,stream_txns,mem_exceeded_count from pg_stat_replication_slots ;
spill_txns | stream_txns | mem_exceeded_count
------------+-------------+--------------------
38 | 20 | 58
(1 row)That probably means that mem_exceeded_count would need to be increased.
What do you think?
Right. But one might argue that if we increment mem_exceeded_count
only when serializing or streaming is actually performed,
mem_exceeded_count would be 0 in the first example and therefore users
would be able to simply check mem_exceeded_count without any
computation.
BTW, while doing some tests for the above examples, I noticed that the patch
was missing a check on memExceededCount in UpdateDecodingStats() (that produced
mem_exceeded_count being 0 for one of the new test in test_decoding): Fixed in
v5 attached.
Thank you for updating the patch!
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Hi,
On Thu, Sep 25, 2025 at 12:14:04PM -0700, Masahiko Sawada wrote:
On Thu, Sep 25, 2025 at 3:17 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:That probably means that mem_exceeded_count would need to be increased.
What do you think?
Right. But one might argue that if we increment mem_exceeded_count
only when serializing or streaming is actually performed,
mem_exceeded_count would be 0 in the first example and therefore users
would be able to simply check mem_exceeded_count without any
computation.
Right but we'd not be able to see when the memory limit has been reached for all
the cases (that would hide the aborted transactions case). I think that with
the current approach we have the best of both world (even if it requires some
computations).
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Hi Bertrand,
Thanks for the patch. The patch overall goods look to me. Just a few small comments:
On Sep 25, 2025, at 18:17, Bertrand Drouvot <bertranddrouvot.pg@gmail.com> wrote:
<v5-0001-Add-mem_exceeded_count-to-pg_stat_replication_slo.patch>
1.
```
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -690,6 +690,9 @@ struct ReorderBuffer
int64 streamCount; /* streaming invocation counter */
int64 streamBytes; /* amount of data decoded */+ /* Number of times logical_decoding_work_mem has been reached */
+ int64 memExceededCount;
```For other metrics, the commented with “Statistics about xxx” above, and line comment after every metric. Maybe use the same style, so that it would be easy to add new metrics in future.
2.
```
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2100,7 +2100,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
Datum
pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_REPLICATION_SLOT_COLS 10
+#define PG_STAT_GET_REPLICATION_SLOT_COLS 11
text *slotname_text = PG_GETARG_TEXT_P(0);
NameData slotname;
TupleDesc tupdesc;
@@ -2125,11 +2125,13 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stream_bytes",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 8, "total_txns",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "mem_exceeded_count",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_bytes",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_txns",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 10, "stats_reset",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 10, "total_bytes",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 11, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
```Is it better to add the new field in the last place?
Say if a client does “select * from pg_stat_get_replication_slit()”, it will just gets an extra column instead of mis-ordered columns.
3.
```
+ <para>
+ Number of times the memory used by logical decoding has exceeded
+ <literal>logical_decoding_work_mem</literal>.
+ </para>
```Feels like “has” is not needed.
Maybe the wording can be simplified as:
Number of times logical decoding exceeded <literal>logical_decoding_work_mem</literal>.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
Hi Evan,
On Fri, Sep 26, 2025 at 02:34:58PM +0800, Chao Li wrote:
Hi Bertrand,
Thanks for the patch. The patch overall goods look to me. Just a few small comments:
Thanks for looking at it!
On Sep 25, 2025, at 18:17, Bertrand Drouvot <bertranddrouvot.pg@gmail.com> wrote:
<v5-0001-Add-mem_exceeded_count-to-pg_stat_replication_slo.patch>
1. ``` --- a/src/include/replication/reorderbuffer.h +++ b/src/include/replication/reorderbuffer.h @@ -690,6 +690,9 @@ struct ReorderBuffer int64 streamCount; /* streaming invocation counter */ int64 streamBytes; /* amount of data decoded */+ /* Number of times logical_decoding_work_mem has been reached */ + int64 memExceededCount; ```For other metrics, the commented with “Statistics about xxx” above, and line comment after every metric. Maybe use the same style, so that it would be easy to add new metrics in future.
I'm not sure: for the moment we have only this stat related to logical_decoding_work_mem,
memory usage. If we add other stats in this area later, we could add a comment
"section" as you suggest.
2. ``` --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -2100,7 +2100,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS) Datum pg_stat_get_replication_slot(PG_FUNCTION_ARGS) { -#define PG_STAT_GET_REPLICATION_SLOT_COLS 10 +#define PG_STAT_GET_REPLICATION_SLOT_COLS 11 text *slotname_text = PG_GETARG_TEXT_P(0); NameData slotname; TupleDesc tupdesc; @@ -2125,11 +2125,13 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS) INT8OID, -1, 0); TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stream_bytes", INT8OID, -1, 0); - TupleDescInitEntry(tupdesc, (AttrNumber) 8, "total_txns", + TupleDescInitEntry(tupdesc, (AttrNumber) 8, "mem_exceeded_count", INT8OID, -1, 0); - TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_bytes", + TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_txns", INT8OID, -1, 0); - TupleDescInitEntry(tupdesc, (AttrNumber) 10, "stats_reset", + TupleDescInitEntry(tupdesc, (AttrNumber) 10, "total_bytes", + INT8OID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 11, "stats_reset", TIMESTAMPTZOID, -1, 0); BlessTupleDesc(tupdesc); ```Is it better to add the new field in the last place?
Say if a client does “select * from pg_stat_get_replication_slit()”, it will just gets an extra column instead of mis-ordered columns.
I think it's good to have the function fields "ordering" matching the view
fields ordering. FWIW, the ordering has been discussed in [1]/messages/by-id/CAJpy0uBskXMq65rvWm8a-KR7cSb_sZH9CPRCnWAQrTOF5fciGw@mail.gmail.com.
3. ``` + <para> + Number of times the memory used by logical decoding has exceeded + <literal>logical_decoding_work_mem</literal>. + </para> ```Feels like “has” is not needed.
It's already done that way in other parts of the documentation:
$ git grep "has exceeded" "*sgml"
doc/src/sgml/maintenance.sgml: vacuum has exceeded the defined insert threshold, which is defined as:
doc/src/sgml/monitoring.sgml: logical decoding to decode changes from WAL has exceeded
doc/src/sgml/monitoring.sgml: from WAL for this slot has exceeded
doc/src/sgml/monitoring.sgml: Number of times the memory used by logical decoding has exceeded
doc/src/sgml/ref/create_subscription.sgml: retention duration has exceeded the
So that looks ok to me (I'm not a native English speaker though).
[1]: /messages/by-id/CAJpy0uBskXMq65rvWm8a-KR7cSb_sZH9CPRCnWAQrTOF5fciGw@mail.gmail.com
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
On Thu, Sep 25, 2025 at 10:26 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
Hi,
On Thu, Sep 25, 2025 at 12:14:04PM -0700, Masahiko Sawada wrote:
On Thu, Sep 25, 2025 at 3:17 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:That probably means that mem_exceeded_count would need to be increased.
What do you think?
Right. But one might argue that if we increment mem_exceeded_count
only when serializing or streaming is actually performed,
mem_exceeded_count would be 0 in the first example and therefore users
would be able to simply check mem_exceeded_count without any
computation.Right but we'd not be able to see when the memory limit has been reached for all
the cases (that would hide the aborted transactions case). I think that with
the current approach we have the best of both world (even if it requires some
computations).
Agreed. It would be better to show a raw statistic so that users can
use the number as they want.
I've made a small comment change and added the commit message to the
v5 patch. I'm going to push the attached patch barring any objection
or review comments.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Attachments:
v6-0001-Add-mem_exceeded_count-column-to-pg_stat_replicat.patchapplication/octet-stream; name=v6-0001-Add-mem_exceeded_count-column-to-pg_stat_replicat.patchDownload
From 3f2cd3628db514c59912eadf51082f86538bb6e9 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Tue, 26 Aug 2025 13:08:13 +0000
Subject: [PATCH v6] Add mem_exceeded_count column to
pg_stat_replication_slots.
This commit introduces a new column mem_exceeded_count to the
pg_stat_replication_slots view. This counter tracks how often the
memory used by logical decoding exceeds the logical_decoding_work_mem
limit. The new statistic helps users determine whether exceeding the
logical_decoding_work_mem limit is a rare occurrences or a frequent
issue, information that wasn't available through existing statistics.
Bumps catversion.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/978D21E8-9D3B-40EA-A4B1-F87BABE7868C@yesql.se
---
contrib/test_decoding/expected/stats.out | 68 +++++++++----------
contrib/test_decoding/sql/stats.sql | 10 +--
doc/src/sgml/monitoring.sgml | 11 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/replication/logical/logical.c | 8 ++-
.../replication/logical/reorderbuffer.c | 7 +-
src/backend/utils/activity/pgstat_replslot.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 19 +++---
src/include/catalog/pg_proc.dat | 6 +-
src/include/pgstat.h | 1 +
src/include/replication/reorderbuffer.h | 3 +
src/test/regress/expected/rules.out | 3 +-
12 files changed, 84 insertions(+), 54 deletions(-)
diff --git a/contrib/test_decoding/expected/stats.out b/contrib/test_decoding/expected/stats.out
index de6dc416130..72fbb270334 100644
--- a/contrib/test_decoding/expected/stats.out
+++ b/contrib/test_decoding/expected/stats.out
@@ -37,12 +37,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | t | t
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | t | t | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
RESET logical_decoding_work_mem;
@@ -53,12 +53,12 @@ SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
-- reset stats for all slots
@@ -68,27 +68,27 @@ SELECT pg_stat_reset_replication_slot(NULL);
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | f | f
- regression_slot_stats3 | t | t | f | f
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | f | f | t
+ regression_slot_stats3 | t | t | f | f | t
(3 rows)
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | mem_exceeded_count | total_txns | total_bytes | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+--------------------+------------+-------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
SELECT pg_stat_reset_replication_slot('do-not-exist');
ERROR: replication slot "do-not-exist" does not exist
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | mem_exceeded_count | total_txns | total_bytes | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+--------------------+------------+-------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
-- spilling the xact
@@ -110,12 +110,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
- slot_name | spill_txns | spill_count
-------------------------+------------+-------------
- regression_slot_stats1 | t | t
- regression_slot_stats2 | f | f
- regression_slot_stats3 | f | f
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, mem_exceeded_count > 0 AS mem_exceeded_count FROM pg_stat_replication_slots;
+ slot_name | spill_txns | spill_count | mem_exceeded_count
+------------------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | t
+ regression_slot_stats2 | f | f | f
+ regression_slot_stats3 | f | f | f
(3 rows)
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
@@ -165,10 +165,10 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
- slot_name | spill_txns | spill_count
----------------------------------+------------+-------------
- regression_slot_stats4_twophase | 0 | 0
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+ slot_name | spill_txns | spill_count | mem_exceeded_count
+---------------------------------+------------+-------------+--------------------
+ regression_slot_stats4_twophase | 0 | 0 | 1
(1 row)
DROP TABLE stats_test;
diff --git a/contrib/test_decoding/sql/stats.sql b/contrib/test_decoding/sql/stats.sql
index a022fe1bf07..9964a8efb87 100644
--- a/contrib/test_decoding/sql/stats.sql
+++ b/contrib/test_decoding/sql/stats.sql
@@ -15,16 +15,16 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats1', NULL,
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats2', NULL, NULL, 'skip-empty-xacts', '1');
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats3', NULL, NULL, 'skip-empty-xacts', '1');
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
RESET logical_decoding_work_mem;
-- reset stats for one slot, others should be unaffected
SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
-- reset stats for all slots
SELECT pg_stat_reset_replication_slot(NULL);
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
@@ -41,7 +41,7 @@ SELECT count(*) FROM pg_logical_slot_peek_changes('regression_slot_stats1', NULL
-- background transaction (say by autovacuum) happens in parallel to the main
-- transaction.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, mem_exceeded_count > 0 AS mem_exceeded_count FROM pg_stat_replication_slots;
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
-- https://postgr.es/m/20210317230447.c7uc4g3vbs4wi32i%40alap3.anarazel.de
@@ -65,7 +65,7 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats4_twophas
-- Verify that the decoding doesn't spill already-aborted transaction's changes.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
DROP TABLE stats_test;
SELECT pg_drop_replication_slot('regression_slot_stats1'),
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 3f4a27a736e..0141c00e666 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1620,6 +1620,17 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>mem_exceeded_count</structfield><type>bigint</type>
+ </para>
+ <para>
+ Number of times the memory used by logical decoding has exceeded
+ <literal>logical_decoding_work_mem</literal>.
+ </para>
+ </entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>total_txns</structfield> <type>bigint</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index c77fa0234bb..b18e7c42d17 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1059,6 +1059,7 @@ CREATE VIEW pg_stat_replication_slots AS
s.stream_txns,
s.stream_count,
s.stream_bytes,
+ s.mem_exceeded_count,
s.total_txns,
s.total_bytes,
s.stats_reset
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c68c0481f42..93ed2eb368e 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1955,10 +1955,11 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
PgStat_StatReplSlotEntry repSlotStat;
/* Nothing to do if we don't have any replication stats to be sent. */
- if (rb->spillBytes <= 0 && rb->streamBytes <= 0 && rb->totalBytes <= 0)
+ if (rb->spillBytes <= 0 && rb->streamBytes <= 0 && rb->totalBytes <= 0 &&
+ rb->memExceededCount <= 0)
return;
- elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
+ elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
rb,
rb->spillTxns,
rb->spillCount,
@@ -1966,6 +1967,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamTxns,
rb->streamCount,
rb->streamBytes,
+ rb->memExceededCount,
rb->totalTxns,
rb->totalBytes);
@@ -1975,6 +1977,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
repSlotStat.stream_txns = rb->streamTxns;
repSlotStat.stream_count = rb->streamCount;
repSlotStat.stream_bytes = rb->streamBytes;
+ repSlotStat.mem_exceeded_count = rb->memExceededCount;
repSlotStat.total_txns = rb->totalTxns;
repSlotStat.total_bytes = rb->totalBytes;
@@ -1986,6 +1989,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamTxns = 0;
rb->streamCount = 0;
rb->streamBytes = 0;
+ rb->memExceededCount = 0;
rb->totalTxns = 0;
rb->totalBytes = 0;
}
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index a5e165fb123..6e72864804e 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -390,6 +390,7 @@ ReorderBufferAllocate(void)
buffer->streamTxns = 0;
buffer->streamCount = 0;
buffer->streamBytes = 0;
+ buffer->memExceededCount = 0;
buffer->totalTxns = 0;
buffer->totalBytes = 0;
@@ -3898,13 +3899,17 @@ static void
ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
{
ReorderBufferTXN *txn;
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024);
+
+ if (memory_limit_reached)
+ rb->memExceededCount += 1;
/*
* Bail out if debug_logical_replication_streaming is buffered and we
* haven't exceeded the memory limit.
*/
if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
- rb->size < logical_decoding_work_mem * (Size) 1024)
+ !memory_limit_reached)
return;
/*
diff --git a/src/backend/utils/activity/pgstat_replslot.c b/src/backend/utils/activity/pgstat_replslot.c
index ccfb11c49bf..d210c261ac6 100644
--- a/src/backend/utils/activity/pgstat_replslot.c
+++ b/src/backend/utils/activity/pgstat_replslot.c
@@ -94,6 +94,7 @@ pgstat_report_replslot(ReplicationSlot *slot, const PgStat_StatReplSlotEntry *re
REPLSLOT_ACC(stream_txns);
REPLSLOT_ACC(stream_count);
REPLSLOT_ACC(stream_bytes);
+ REPLSLOT_ACC(mem_exceeded_count);
REPLSLOT_ACC(total_txns);
REPLSLOT_ACC(total_bytes);
#undef REPLSLOT_ACC
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index c756c2bebaa..e64dd5e043e 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2100,7 +2100,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
Datum
pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_REPLICATION_SLOT_COLS 10
+#define PG_STAT_GET_REPLICATION_SLOT_COLS 11
text *slotname_text = PG_GETARG_TEXT_P(0);
NameData slotname;
TupleDesc tupdesc;
@@ -2125,11 +2125,13 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stream_bytes",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 8, "total_txns",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "mem_exceeded_count",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_bytes",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_txns",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 10, "stats_reset",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 10, "total_bytes",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 11, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
@@ -2152,13 +2154,14 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
values[4] = Int64GetDatum(slotent->stream_txns);
values[5] = Int64GetDatum(slotent->stream_count);
values[6] = Int64GetDatum(slotent->stream_bytes);
- values[7] = Int64GetDatum(slotent->total_txns);
- values[8] = Int64GetDatum(slotent->total_bytes);
+ values[7] = Int64GetDatum(slotent->mem_exceeded_count);
+ values[8] = Int64GetDatum(slotent->total_txns);
+ values[9] = Int64GetDatum(slotent->total_bytes);
if (slotent->stat_reset_timestamp == 0)
- nulls[9] = true;
+ nulls[10] = true;
else
- values[9] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
+ values[10] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 01eba3b5a19..4ea72c85240 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5687,9 +5687,9 @@
{ oid => '6169', descr => 'statistics: information about replication slot',
proname => 'pg_stat_get_replication_slot', provolatile => 's',
proparallel => 'r', prorettype => 'record', proargtypes => 'text',
- proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
+ proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,mem_exceeded_count,total_txns,total_bytes,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
{ oid => '6230', descr => 'statistics: check if a stats object exists',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index e4a59a30b8c..737aa1cc551 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -394,6 +394,7 @@ typedef struct PgStat_StatReplSlotEntry
PgStat_Counter stream_txns;
PgStat_Counter stream_count;
PgStat_Counter stream_bytes;
+ PgStat_Counter mem_exceeded_count;
PgStat_Counter total_txns;
PgStat_Counter total_bytes;
TimestampTz stat_reset_timestamp;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 91dc7e5e448..3cbe106a3c7 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -690,6 +690,9 @@ struct ReorderBuffer
int64 streamCount; /* streaming invocation counter */
int64 streamBytes; /* amount of data decoded */
+ /* Number of times the logical_decoding_work_mem limit has been reached */
+ int64 memExceededCount;
+
/*
* Statistics about all the transactions sent to the decoding output
* plugin
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 35e8aad7701..f4f3a2a3018 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2138,11 +2138,12 @@ pg_stat_replication_slots| SELECT s.slot_name,
s.stream_txns,
s.stream_count,
s.stream_bytes,
+ s.mem_exceeded_count,
s.total_txns,
s.total_bytes,
s.stats_reset
FROM pg_replication_slots r,
- LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, total_txns, total_bytes, stats_reset)
+ LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, mem_exceeded_count, total_txns, total_bytes, stats_reset)
WHERE (r.datoid IS NOT NULL);
pg_stat_slru| SELECT name,
blks_zeroed,
--
2.47.3
Hi,
On Thu, Oct 02, 2025 at 04:39:40PM -0700, Masahiko Sawada wrote:
On Thu, Sep 25, 2025 at 10:26 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Agreed. It would be better to show a raw statistic so that users can
use the number as they want.I've made a small comment change
Thanks!
Comparing v5 and v6:
- /* Number of times logical_decoding_work_mem has been reached */
+ /* Number of times the logical_decoding_work_mem limit has been reached */LGTM.
and added the commit message to the
v5 patch. I'm going to push the attached patch barring any objection
or review comments.
The commit message LGTM.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
On Fri, Oct 3, 2025 at 5:10 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Thu, Sep 25, 2025 at 10:26 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Hi,
On Thu, Sep 25, 2025 at 12:14:04PM -0700, Masahiko Sawada wrote:
On Thu, Sep 25, 2025 at 3:17 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:That probably means that mem_exceeded_count would need to be increased.
What do you think?
Right. But one might argue that if we increment mem_exceeded_count
only when serializing or streaming is actually performed,
mem_exceeded_count would be 0 in the first example and therefore users
would be able to simply check mem_exceeded_count without any
computation.Right but we'd not be able to see when the memory limit has been reached for all
the cases (that would hide the aborted transactions case). I think that with
the current approach we have the best of both world (even if it requires some
computations).Agreed. It would be better to show a raw statistic so that users can
use the number as they want.I've made a small comment change and added the commit message to the
v5 patch. I'm going to push the attached patch barring any objection
or review comments.
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem *
(Size) 1024);
+
+ if (memory_limit_reached)
+ rb->memExceededCount += 1;If the memory limit is hit but no transaction was serialized, the
stats won't be updated since UpdateDecodingStats() won't be called. We
need to call UpdateDecodingStats() in ReorderBufferCheckMemoryLimit()
if no transaction was streamed or spilled.
-SELECT slot_name, spill_txns, spill_count FROM
pg_stat_replication_slots WHERE slot_name =
'regression_slot_stats4_twophase';
- slot_name | spill_txns | spill_count
----------------------------------+------------+-------------
- regression_slot_stats4_twophase | 0 | 0
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM
pg_stat_replication_slots WHERE slot_name =
'regression_slot_stats4_twophase';
+ slot_name | spill_txns | spill_count |
mem_exceeded_count
+---------------------------------+------------+-------------+--------------------
+ regression_slot_stats4_twophase | 0 | 0 |
1
(1 row)Are we sure that mem_exceeded_count will always be 1 in this case? Can
it be 2 or more because of background activity?
--
Best Wishes,
Ashutosh Bapat
Hi,
On Fri, Oct 03, 2025 at 05:19:42PM +0530, Ashutosh Bapat wrote:
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024); + + if (memory_limit_reached) + rb->memExceededCount += 1;
Thanks for looking at it!
If the memory limit is hit but no transaction was serialized, the
stats won't be updated since UpdateDecodingStats() won't be called. We
need to call UpdateDecodingStats() in ReorderBufferCheckMemoryLimit()
if no transaction was streamed or spilled.
I did some testing and the stats are reported because UpdateDecodingStats() is
also called in DecodeCommit(), DecodeAbort() and DecodePrepare() (in addition
to ReorderBufferSerializeTXN() and ReorderBufferStreamTXN()). That's also why
,for example, total_txns is reported even if no transaction was streamed or
spilled.
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase'; - slot_name | spill_txns | spill_count ----------------------------------+------------+------------- - regression_slot_stats4_twophase | 0 | 0 +SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase'; + slot_name | spill_txns | spill_count | mem_exceeded_count +---------------------------------+------------+-------------+-------------------- + regression_slot_stats4_twophase | 0 | 0 | 1 (1 row)Are we sure that mem_exceeded_count will always be 1 in this case? Can
it be 2 or more because of background activity?
I think that the question could be the same for spill_txns and spill_count. It
seems to have been working fine (that way) since this test exists (added in
072ee847ad4c) but I think that you raised a good point.
Sawada-San, what do you think about this particular test, is it safe to rely
on the exact values here?
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
On Fri, Oct 3, 2025 at 6:45 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
Hi,
On Fri, Oct 03, 2025 at 05:19:42PM +0530, Ashutosh Bapat wrote:
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024); + + if (memory_limit_reached) + rb->memExceededCount += 1;Thanks for looking at it!
If the memory limit is hit but no transaction was serialized, the
stats won't be updated since UpdateDecodingStats() won't be called. We
need to call UpdateDecodingStats() in ReorderBufferCheckMemoryLimit()
if no transaction was streamed or spilled.I did some testing and the stats are reported because UpdateDecodingStats() is
also called in DecodeCommit(), DecodeAbort() and DecodePrepare() (in addition
to ReorderBufferSerializeTXN() and ReorderBufferStreamTXN()). That's also why
,for example, total_txns is reported even if no transaction was streamed or
spilled.
In a very pathological case, where all transactions happen to be
aborted while decoding and yet memory limit is hit many times, nothing
will be reported till first committed transaction after it is decoded.
Which may never happen. I didn't find a call stack where by
UpdateDecodingStats could be reached from
ReorderBufferCheckAndTruncateAbortedTXN().
--
Best Wishes,
Ashutosh Bapat
On Fri, Oct 3, 2025 at 6:15 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
Hi,
On Fri, Oct 03, 2025 at 05:19:42PM +0530, Ashutosh Bapat wrote:
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024); + + if (memory_limit_reached) + rb->memExceededCount += 1;Thanks for looking at it!
+1
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase'; - slot_name | spill_txns | spill_count ----------------------------------+------------+------------- - regression_slot_stats4_twophase | 0 | 0 +SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase'; + slot_name | spill_txns | spill_count | mem_exceeded_count +---------------------------------+------------+-------------+-------------------- + regression_slot_stats4_twophase | 0 | 0 | 1 (1 row)Are we sure that mem_exceeded_count will always be 1 in this case? Can
it be 2 or more because of background activity?I think that the question could be the same for spill_txns and spill_count. It
seems to have been working fine (that way) since this test exists (added in
072ee847ad4c) but I think that you raised a good point.Sawada-San, what do you think about this particular test, is it safe to rely
on the exact values here?
In short, I'm fine with the change proposed by Ashtosh. I believe that
in this case it's safe to rely on the exact values in principle since
once we reach the memory limit we truncate all changes in the
transaction and skip further changes. This test with the patch fails
if there are other activities enough to reach the memory limit and
those transactions are aborted, which it's unlikely to happen, I
guess. That being said, there is no downside if we check
'mem_exceeded_count > 0' instead of checking the exact number and it
seems more stable for future changes.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Fri, Oct 3, 2025 at 9:26 AM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
On Fri, Oct 3, 2025 at 6:45 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Hi,
On Fri, Oct 03, 2025 at 05:19:42PM +0530, Ashutosh Bapat wrote:
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024); + + if (memory_limit_reached) + rb->memExceededCount += 1;Thanks for looking at it!
If the memory limit is hit but no transaction was serialized, the
stats won't be updated since UpdateDecodingStats() won't be called. We
need to call UpdateDecodingStats() in ReorderBufferCheckMemoryLimit()
if no transaction was streamed or spilled.I did some testing and the stats are reported because UpdateDecodingStats() is
also called in DecodeCommit(), DecodeAbort() and DecodePrepare() (in addition
to ReorderBufferSerializeTXN() and ReorderBufferStreamTXN()). That's also why
,for example, total_txns is reported even if no transaction was streamed or
spilled.In a very pathological case, where all transactions happen to be
aborted while decoding and yet memory limit is hit many times, nothing
will be reported till first committed transaction after it is decoded.
Which may never happen. I didn't find a call stack where by
UpdateDecodingStats could be reached from
ReorderBufferCheckAndTruncateAbortedTXN().
The more we report the status frequently, the less chances we lose the
statistics in case of logical decoding being interrupted but the more
overheads we have to update the statistics. I personally prefer not to
call UpdateDecodingStats() frequently since pgstat_report_replslot()
always flush the statistics. If the transaction is serialized or
streamed, we can update the memExceededCount together with other
statistics such as streamBytes and spillBytes. But if we can free
enough memory only by truncating already-aborted transactions, we need
to rely on the next committed/aborted/prepared transaction to update
the statistics. So how about calling UpdateDecodingStats() only in
case where we only truncate aborted transactions and the memory usage
gets lower than the limit?
I've attached the patch that implements this idea with a small
refactoring. It also has the change to the regression test results
we've discussed.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Attachments:
fix_masahiko.patch.txttext/plain; charset=US-ASCII; name=fix_masahiko.patch.txtDownload
diff --git a/contrib/test_decoding/expected/stats.out b/contrib/test_decoding/expected/stats.out
index 72fbb270334..13d2be24e18 100644
--- a/contrib/test_decoding/expected/stats.out
+++ b/contrib/test_decoding/expected/stats.out
@@ -165,10 +165,10 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count > 0 as mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
slot_name | spill_txns | spill_count | mem_exceeded_count
---------------------------------+------------+-------------+--------------------
- regression_slot_stats4_twophase | 0 | 0 | 1
+ regression_slot_stats4_twophase | 0 | 0 | t
(1 row)
DROP TABLE stats_test;
diff --git a/contrib/test_decoding/sql/stats.sql b/contrib/test_decoding/sql/stats.sql
index 9964a8efb87..b9aae0c7b63 100644
--- a/contrib/test_decoding/sql/stats.sql
+++ b/contrib/test_decoding/sql/stats.sql
@@ -65,7 +65,7 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats4_twophas
-- Verify that the decoding doesn't spill already-aborted transaction's changes.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count > 0 as mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
DROP TABLE stats_test;
SELECT pg_drop_replication_slot('regression_slot_stats1'),
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 6e72864804e..094b928cf35 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3899,18 +3899,26 @@ static void
ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
{
ReorderBufferTXN *txn;
- bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024);
+ bool update_stats = true;
- if (memory_limit_reached)
+ if (rb->size >= logical_decoding_work_mem * (Size) 1024)
+ {
+ /*
+ * Update the statistics as the memory usage has reached the limit. We
+ * report the statistics update later in this function since we can
+ * update the slot statistics altogether while streaming or
+ * serializing transactions in most cases.
+ */
rb->memExceededCount += 1;
-
- /*
- * Bail out if debug_logical_replication_streaming is buffered and we
- * haven't exceeded the memory limit.
- */
- if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
- !memory_limit_reached)
+ }
+ else if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED)
+ {
+ /*
+ * Bail out if debug_logical_replication_streaming is buffered and we
+ * haven't exceeded the memory limit.
+ */
return;
+ }
/*
* If debug_logical_replication_streaming is immediate, loop until there's
@@ -3970,8 +3978,14 @@ ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
*/
Assert(txn->size == 0);
Assert(txn->nentries_mem == 0);
+
+ /* We've reported memExceededCount update */
+ update_stats = false;
}
+ if (update_stats)
+ UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
+
/* We must be under the memory limit now. */
Assert(rb->size < logical_decoding_work_mem * (Size) 1024);
}
On Fri, Oct 3, 2025 at 11:48 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Fri, Oct 3, 2025 at 9:26 AM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:On Fri, Oct 3, 2025 at 6:45 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Hi,
On Fri, Oct 03, 2025 at 05:19:42PM +0530, Ashutosh Bapat wrote:
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024); + + if (memory_limit_reached) + rb->memExceededCount += 1;Thanks for looking at it!
If the memory limit is hit but no transaction was serialized, the
stats won't be updated since UpdateDecodingStats() won't be called. We
need to call UpdateDecodingStats() in ReorderBufferCheckMemoryLimit()
if no transaction was streamed or spilled.I did some testing and the stats are reported because UpdateDecodingStats() is
also called in DecodeCommit(), DecodeAbort() and DecodePrepare() (in addition
to ReorderBufferSerializeTXN() and ReorderBufferStreamTXN()). That's also why
,for example, total_txns is reported even if no transaction was streamed or
spilled.In a very pathological case, where all transactions happen to be
aborted while decoding and yet memory limit is hit many times, nothing
will be reported till first committed transaction after it is decoded.
Which may never happen. I didn't find a call stack where by
UpdateDecodingStats could be reached from
ReorderBufferCheckAndTruncateAbortedTXN().The more we report the status frequently, the less chances we lose the
statistics in case of logical decoding being interrupted but the more
overheads we have to update the statistics. I personally prefer not to
call UpdateDecodingStats() frequently since pgstat_report_replslot()
always flush the statistics. If the transaction is serialized or
streamed, we can update the memExceededCount together with other
statistics such as streamBytes and spillBytes. But if we can free
enough memory only by truncating already-aborted transactions, we need
to rely on the next committed/aborted/prepared transaction to update
the statistics. So how about calling UpdateDecodingStats() only in
case where we only truncate aborted transactions and the memory usage
gets lower than the limit?
Yes, that's what my intention was.
I've attached the patch that implements this idea with a small
refactoring. It also has the change to the regression test results
we've discussed.
The change looks good to me.
Given Andres's comment, in a nearby thread, about being cautious about
adding useless statistics, I think this one needs a bit more
discussion. In the proposal email Bertant wrote
Please find attached a patch to $SUBJECT, to report the number of times the
logical_decoding_work_mem has been reached.With such a counter one could get a ratio like total_txns/memory_limit_hits.
That could help to see if reaching logical_decoding_work_mem is rare or
frequent enough. If frequent, then maybe there is a need to adjust
logical_decoding_work_mem.
I agree with the goal that we need a metric to decide whether
exceeding logical decoding work mem is frequent or not.
Based on my simple example above, one could say that it might be possible to get
the same with:(spill_count - spill_txns) + (stream_count - stream_txns)
but that doesn't appear to be the case with a more complicated example (277 vs 247):
slot_name | spill_txns | spill_count | total_txns | stream_txns | stream_count | memory_limit_hits | (spc-spct)+(strc-strt)
--------------+------------+-------------+------------+-------------+--------------+-------------------+------------------------
logical_slot | 405 | 552 | 19 | 5 | 105 | 277 | 247
(1 row)
Agreed that any arithmetic on the currently reported counters don't
provide the exact number of times the memory limit was hit. The
question is whether there exists some arithmetic which gives a good
indicator of whether hitting memory limit is frequent or rare. In your
example, is the difference 247 vs 277 significant enough to lead to
the wrong conclusion about the frequency? Is there another case where
this difference is going to lead to a wrong conclusion?
--
Best Wishes,
Ashutosh Bapat
Hi,
On Mon, Oct 06, 2025 at 10:50:52AM +0530, Ashutosh Bapat wrote:
On Fri, Oct 3, 2025 at 11:48 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Fri, Oct 3, 2025 at 9:26 AM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:On Fri, Oct 3, 2025 at 6:45 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Hi,
On Fri, Oct 03, 2025 at 05:19:42PM +0530, Ashutosh Bapat wrote:
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024); + + if (memory_limit_reached) + rb->memExceededCount += 1;Thanks for looking at it!
If the memory limit is hit but no transaction was serialized, the
stats won't be updated since UpdateDecodingStats() won't be called. We
need to call UpdateDecodingStats() in ReorderBufferCheckMemoryLimit()
if no transaction was streamed or spilled.I did some testing and the stats are reported because UpdateDecodingStats() is
also called in DecodeCommit(), DecodeAbort() and DecodePrepare() (in addition
to ReorderBufferSerializeTXN() and ReorderBufferStreamTXN()). That's also why
,for example, total_txns is reported even if no transaction was streamed or
spilled.In a very pathological case, where all transactions happen to be
aborted while decoding and yet memory limit is hit many times, nothing
will be reported till first committed transaction after it is decoded.
Which may never happen. I didn't find a call stack where by
UpdateDecodingStats could be reached from
ReorderBufferCheckAndTruncateAbortedTXN().The more we report the status frequently, the less chances we lose the
statistics in case of logical decoding being interrupted but the more
overheads we have to update the statistics. I personally prefer not to
call UpdateDecodingStats() frequently since pgstat_report_replslot()
always flush the statistics. If the transaction is serialized or
streamed, we can update the memExceededCount together with other
statistics such as streamBytes and spillBytes. But if we can free
enough memory only by truncating already-aborted transactions, we need
to rely on the next committed/aborted/prepared transaction to update
the statistics.
Indeed, there is cases when committed/aborted/prepared would not be called
right after ReorderBufferCheckAndTruncateAbortedTXN().
So how about calling UpdateDecodingStats() only in
case where we only truncate aborted transactions and the memory usage
gets lower than the limit?Yes, that's what my intention was.
I also think it makes sense.
I've attached the patch that implements this idea with a small
refactoring.
Thanks!
It also has the change to the regression test results
we've discussed.
-SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count > 0 as mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
slot_name | spill_txns | spill_count | mem_exceeded_count
---------------------------------+------------+-------------+--------------------
- regression_slot_stats4_twophase | 0 | 0 | 1
+ regression_slot_stats4_twophase | 0 | 0 | tCould we also imagine that there are other activities enough to reach the memory
limit and transactions are not aborted, meaning spill_txns and/or spill_count are > 0?
In that case we may want to get rid of this test instead (as checking spill_txns >=0
and spill_count >=0 would not really reflect the intend of this test).
The change looks good to me.
Given Andres's comment, in a nearby thread, about being cautious about
adding useless statistics, I think this one needs a bit more
discussion. In the proposal email Bertant wrotePlease find attached a patch to $SUBJECT, to report the number of times the
logical_decoding_work_mem has been reached.With such a counter one could get a ratio like total_txns/memory_limit_hits.
That could help to see if reaching logical_decoding_work_mem is rare or
frequent enough. If frequent, then maybe there is a need to adjust
logical_decoding_work_mem.I agree with the goal that we need a metric to decide whether
exceeding logical decoding work mem is frequent or not.Based on my simple example above, one could say that it might be possible to get
the same with:(spill_count - spill_txns) + (stream_count - stream_txns)
but that doesn't appear to be the case with a more complicated example (277 vs 247):
slot_name | spill_txns | spill_count | total_txns | stream_txns | stream_count | memory_limit_hits | (spc-spct)+(strc-strt)
--------------+------------+-------------+------------+-------------+--------------+-------------------+------------------------
logical_slot | 405 | 552 | 19 | 5 | 105 | 277 | 247
(1 row)Is there another case where this difference is going to lead to a wrong conclusion?
Yeah, for example when the ratio aborted/committed is high, we could get things
like:
slot_name | spill_txns | spill_count | stream_txns | stream_count | total_txns | mem_exceeded_count | (spc-spct)+(strc-strt)
--------------+------------+-------------+-------------+--------------+------------+--------------------+------------------------
logical_slot | 1 | 2 | 0 | 0 | 192 | 244 | 1
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
On Sun, Oct 5, 2025 at 11:52 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
Hi,
On Mon, Oct 06, 2025 at 10:50:52AM +0530, Ashutosh Bapat wrote:
On Fri, Oct 3, 2025 at 11:48 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Fri, Oct 3, 2025 at 9:26 AM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:On Fri, Oct 3, 2025 at 6:45 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Hi,
On Fri, Oct 03, 2025 at 05:19:42PM +0530, Ashutosh Bapat wrote:
+ bool memory_limit_reached = (rb->size >= logical_decoding_work_mem * (Size) 1024); + + if (memory_limit_reached) + rb->memExceededCount += 1;Thanks for looking at it!
If the memory limit is hit but no transaction was serialized, the
stats won't be updated since UpdateDecodingStats() won't be called. We
need to call UpdateDecodingStats() in ReorderBufferCheckMemoryLimit()
if no transaction was streamed or spilled.I did some testing and the stats are reported because UpdateDecodingStats() is
also called in DecodeCommit(), DecodeAbort() and DecodePrepare() (in addition
to ReorderBufferSerializeTXN() and ReorderBufferStreamTXN()). That's also why
,for example, total_txns is reported even if no transaction was streamed or
spilled.In a very pathological case, where all transactions happen to be
aborted while decoding and yet memory limit is hit many times, nothing
will be reported till first committed transaction after it is decoded.
Which may never happen. I didn't find a call stack where by
UpdateDecodingStats could be reached from
ReorderBufferCheckAndTruncateAbortedTXN().The more we report the status frequently, the less chances we lose the
statistics in case of logical decoding being interrupted but the more
overheads we have to update the statistics. I personally prefer not to
call UpdateDecodingStats() frequently since pgstat_report_replslot()
always flush the statistics. If the transaction is serialized or
streamed, we can update the memExceededCount together with other
statistics such as streamBytes and spillBytes. But if we can free
enough memory only by truncating already-aborted transactions, we need
to rely on the next committed/aborted/prepared transaction to update
the statistics.Indeed, there is cases when committed/aborted/prepared would not be called
right after ReorderBufferCheckAndTruncateAbortedTXN().So how about calling UpdateDecodingStats() only in
case where we only truncate aborted transactions and the memory usage
gets lower than the limit?Yes, that's what my intention was.
I also think it makes sense.
I've attached the patch that implements this idea with a small
refactoring.Thanks!
It also has the change to the regression test results
we've discussed.
-SELECT slot_name, spill_txns, spill_count, mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase'; +SELECT slot_name, spill_txns, spill_count, mem_exceeded_count > 0 as mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase'; slot_name | spill_txns | spill_count | mem_exceeded_count ---------------------------------+------------+-------------+-------------------- - regression_slot_stats4_twophase | 0 | 0 | 1 + regression_slot_stats4_twophase | 0 | 0 | tCould we also imagine that there are other activities enough to reach the memory
limit and transactions are not aborted, meaning spill_txns and/or spill_count are > 0?In that case we may want to get rid of this test instead (as checking spill_txns >=0
and spill_count >=0 would not really reflect the intend of this test).
It makes sense to me to make an assumption that there are no
concurrent activities that are capturable by logical decoding during
this test. So I think we don't need to care about that case. On the
other hand, under this assumption, it also makes sense to check it
with the exact number. I've chosen >0 as we can achieve the same goal
while being more flexible for potential future changes. I'm open to
other suggestions though.
The change looks good to me.
Given Andres's comment, in a nearby thread, about being cautious about
adding useless statistics, I think this one needs a bit more
discussion. In the proposal email Bertant wrotePlease find attached a patch to $SUBJECT, to report the number of times the
logical_decoding_work_mem has been reached.With such a counter one could get a ratio like total_txns/memory_limit_hits.
That could help to see if reaching logical_decoding_work_mem is rare or
frequent enough. If frequent, then maybe there is a need to adjust
logical_decoding_work_mem.I agree with the goal that we need a metric to decide whether
exceeding logical decoding work mem is frequent or not.Based on my simple example above, one could say that it might be possible to get
the same with:(spill_count - spill_txns) + (stream_count - stream_txns)
but that doesn't appear to be the case with a more complicated example (277 vs 247):
slot_name | spill_txns | spill_count | total_txns | stream_txns | stream_count | memory_limit_hits | (spc-spct)+(strc-strt)
--------------+------------+-------------+------------+-------------+--------------+-------------------+------------------------
logical_slot | 405 | 552 | 19 | 5 | 105 | 277 | 247
(1 row)Is there another case where this difference is going to lead to a wrong conclusion?
Yeah, for example when the ratio aborted/committed is high, we could get things
like:slot_name | spill_txns | spill_count | stream_txns | stream_count | total_txns | mem_exceeded_count | (spc-spct)+(strc-strt)
--------------+------------+-------------+-------------+--------------+------------+--------------------+------------------------
logical_slot | 1 | 2 | 0 | 0 | 192 | 244 | 1
+1
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Hi,
On Mon, Oct 06, 2025 at 01:18:38PM -0700, Masahiko Sawada wrote:
On Sun, Oct 5, 2025 at 11:52 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Could we also imagine that there are other activities enough to reach the memory
limit and transactions are not aborted, meaning spill_txns and/or spill_count are > 0?In that case we may want to get rid of this test instead (as checking spill_txns >=0
and spill_count >=0 would not really reflect the intend of this test).It makes sense to me to make an assumption that there are no
concurrent activities that are capturable by logical decoding during
this test. So I think we don't need to care about that case. On the
other hand, under this assumption, it also makes sense to check it
with the exact number. I've chosen >0 as we can achieve the same goal
while being more flexible for potential future changes. I'm open to
other suggestions though.
0 is fine by me. I was just wondering about spill_txns and spill_count too.
That could sound weird that we are confident for spill_txns and spill_count
to rely on the exact values and not for the new field. That said, I agree that
0 is more flexible for potential future changes (in the sense that this one
is more likely to change in its implementation). In short, I'm fine with your
proposal.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
On Tue, Oct 7, 2025 at 1:08 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
Hi,
On Mon, Oct 06, 2025 at 01:18:38PM -0700, Masahiko Sawada wrote:
On Sun, Oct 5, 2025 at 11:52 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:Could we also imagine that there are other activities enough to reach the memory
limit and transactions are not aborted, meaning spill_txns and/or spill_count are > 0?In that case we may want to get rid of this test instead (as checking spill_txns >=0
and spill_count >=0 would not really reflect the intend of this test).It makes sense to me to make an assumption that there are no
concurrent activities that are capturable by logical decoding during
this test. So I think we don't need to care about that case. On the
other hand, under this assumption, it also makes sense to check it
with the exact number. I've chosen >0 as we can achieve the same goal
while being more flexible for potential future changes. I'm open to
other suggestions though.0 is fine by me. I was just wondering about spill_txns and spill_count too.
That could sound weird that we are confident for spill_txns and spill_count
to rely on the exact values and not for the new field. That said, I agree that0 is more flexible for potential future changes (in the sense that this one
is more likely to change in its implementation). In short, I'm fine with your
proposal.
Thank you for the comment. I've noted this discussion as a comment in
the new tests.
I've attached the updated version patch. Please review it.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Attachments:
v7-0001-Add-mem_exceeded_count-column-to-pg_stat_replicat.patchapplication/octet-stream; name=v7-0001-Add-mem_exceeded_count-column-to-pg_stat_replicat.patchDownload
From cb5353fdcd5697ce8f36a1dd25f8d1481cab55f2 Mon Sep 17 00:00:00 2001
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Date: Tue, 26 Aug 2025 13:08:13 +0000
Subject: [PATCH v7] Add mem_exceeded_count column to
pg_stat_replication_slots.
This commit introduces a new column mem_exceeded_count to the
pg_stat_replication_slots view. This counter tracks how often the
memory used by logical decoding exceeds the logical_decoding_work_mem
limit. The new statistic helps users determine whether exceeding the
logical_decoding_work_mem limit is a rare occurrences or a frequent
issue, information that wasn't available through existing statistics.
Bumps catversion.
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/978D21E8-9D3B-40EA-A4B1-F87BABE7868C@yesql.se
---
contrib/test_decoding/expected/stats.out | 71 ++++++++++---------
contrib/test_decoding/sql/stats.sql | 13 ++--
doc/src/sgml/monitoring.sgml | 11 +++
src/backend/catalog/system_views.sql | 1 +
src/backend/replication/logical/logical.c | 8 ++-
.../replication/logical/reorderbuffer.c | 34 +++++++--
src/backend/utils/activity/pgstat_replslot.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 19 ++---
src/include/catalog/pg_proc.dat | 6 +-
src/include/pgstat.h | 1 +
src/include/replication/reorderbuffer.h | 3 +
src/test/regress/expected/rules.out | 3 +-
12 files changed, 112 insertions(+), 59 deletions(-)
diff --git a/contrib/test_decoding/expected/stats.out b/contrib/test_decoding/expected/stats.out
index de6dc416130..28da9123cc8 100644
--- a/contrib/test_decoding/expected/stats.out
+++ b/contrib/test_decoding/expected/stats.out
@@ -37,12 +37,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | t | t
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | t | t | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
RESET logical_decoding_work_mem;
@@ -53,12 +53,12 @@ SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | t | t
- regression_slot_stats3 | t | t | t | t
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | t | t | t
+ regression_slot_stats3 | t | t | t | t | t
(3 rows)
-- reset stats for all slots
@@ -68,27 +68,27 @@ SELECT pg_stat_reset_replication_slot(NULL);
(1 row)
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
- slot_name | spill_txns | spill_count | total_txns | total_bytes
-------------------------+------------+-------------+------------+-------------
- regression_slot_stats1 | t | t | f | f
- regression_slot_stats2 | t | t | f | f
- regression_slot_stats3 | t | t | f | f
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
+ slot_name | spill_txns | spill_count | total_txns | total_bytes | mem_exceeded_count
+------------------------+------------+-------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | f | f | t
+ regression_slot_stats2 | t | t | f | f | t
+ regression_slot_stats3 | t | t | f | f | t
(3 rows)
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | mem_exceeded_count | total_txns | total_bytes | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+--------------------+------------+-------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
SELECT pg_stat_reset_replication_slot('do-not-exist');
ERROR: replication slot "do-not-exist" does not exist
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
- slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | total_txns | total_bytes | stats_reset
---------------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+-------------
- do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+ slot_name | spill_txns | spill_count | spill_bytes | stream_txns | stream_count | stream_bytes | mem_exceeded_count | total_txns | total_bytes | stats_reset
+--------------+------------+-------------+-------------+-------------+--------------+--------------+--------------------+------------+-------------+-------------
+ do-not-exist | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(1 row)
-- spilling the xact
@@ -110,12 +110,12 @@ SELECT pg_stat_force_next_flush();
(1 row)
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
- slot_name | spill_txns | spill_count
-------------------------+------------+-------------
- regression_slot_stats1 | t | t
- regression_slot_stats2 | f | f
- regression_slot_stats3 | f | f
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, mem_exceeded_count > 0 AS mem_exceeded_count FROM pg_stat_replication_slots;
+ slot_name | spill_txns | spill_count | mem_exceeded_count
+------------------------+------------+-------------+--------------------
+ regression_slot_stats1 | t | t | t
+ regression_slot_stats2 | f | f | f
+ regression_slot_stats3 | f | f | f
(3 rows)
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
@@ -159,16 +159,19 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats4_twophas
(1 row)
-- Verify that the decoding doesn't spill already-aborted transaction's changes.
+-- Given that there is no concurrent activities that are capturable by logical decoding,
+-- mem_exceeded_count should theoretically be 1 but we check if >0 here since it's
+-- more flexible for potential future changes and adequate for the testing purpose.
SELECT pg_stat_force_next_flush();
pg_stat_force_next_flush
--------------------------
(1 row)
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
- slot_name | spill_txns | spill_count
----------------------------------+------------+-------------
- regression_slot_stats4_twophase | 0 | 0
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count > 0 as mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+ slot_name | spill_txns | spill_count | mem_exceeded_count
+---------------------------------+------------+-------------+--------------------
+ regression_slot_stats4_twophase | 0 | 0 | t
(1 row)
DROP TABLE stats_test;
diff --git a/contrib/test_decoding/sql/stats.sql b/contrib/test_decoding/sql/stats.sql
index a022fe1bf07..6661dbcb85c 100644
--- a/contrib/test_decoding/sql/stats.sql
+++ b/contrib/test_decoding/sql/stats.sql
@@ -15,16 +15,16 @@ SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats1', NULL,
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats2', NULL, NULL, 'skip-empty-xacts', '1');
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats3', NULL, NULL, 'skip-empty-xacts', '1');
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
RESET logical_decoding_work_mem;
-- reset stats for one slot, others should be unaffected
SELECT pg_stat_reset_replication_slot('regression_slot_stats1');
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
-- reset stats for all slots
SELECT pg_stat_reset_replication_slot(NULL);
-SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes FROM pg_stat_replication_slots ORDER BY slot_name;
+SELECT slot_name, spill_txns = 0 AS spill_txns, spill_count = 0 AS spill_count, total_txns > 0 AS total_txns, total_bytes > 0 AS total_bytes, mem_exceeded_count = 0 AS mem_exceeded_count FROM pg_stat_replication_slots ORDER BY slot_name;
-- verify accessing/resetting stats for non-existent slot does something reasonable
SELECT * FROM pg_stat_get_replication_slot('do-not-exist');
@@ -41,7 +41,7 @@ SELECT count(*) FROM pg_logical_slot_peek_changes('regression_slot_stats1', NULL
-- background transaction (say by autovacuum) happens in parallel to the main
-- transaction.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count FROM pg_stat_replication_slots;
+SELECT slot_name, spill_txns > 0 AS spill_txns, spill_count > 0 AS spill_count, mem_exceeded_count > 0 AS mem_exceeded_count FROM pg_stat_replication_slots;
-- Ensure stats can be repeatedly accessed using the same stats snapshot. See
-- https://postgr.es/m/20210317230447.c7uc4g3vbs4wi32i%40alap3.anarazel.de
@@ -64,8 +64,11 @@ ROLLBACK PREPARED 'test1_abort';
SELECT count(*) FROM pg_logical_slot_get_changes('regression_slot_stats4_twophase', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
-- Verify that the decoding doesn't spill already-aborted transaction's changes.
+-- Given that there is no concurrent activities that are capturable by logical decoding,
+-- mem_exceeded_count should theoretically be 1 but we check if >0 here since it's
+-- more flexible for potential future changes and adequate for the testing purpose.
SELECT pg_stat_force_next_flush();
-SELECT slot_name, spill_txns, spill_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
+SELECT slot_name, spill_txns, spill_count, mem_exceeded_count > 0 as mem_exceeded_count FROM pg_stat_replication_slots WHERE slot_name = 'regression_slot_stats4_twophase';
DROP TABLE stats_test;
SELECT pg_drop_replication_slot('regression_slot_stats1'),
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 6e3aac3d815..595c8245f22 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -1620,6 +1620,17 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>mem_exceeded_count</structfield><type>bigint</type>
+ </para>
+ <para>
+ Number of times the memory used by logical decoding has exceeded
+ <literal>logical_decoding_work_mem</literal>.
+ </para>
+ </entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>total_txns</structfield> <type>bigint</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 884b6a23817..fba6e171883 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1063,6 +1063,7 @@ CREATE VIEW pg_stat_replication_slots AS
s.stream_txns,
s.stream_count,
s.stream_bytes,
+ s.mem_exceeded_count,
s.total_txns,
s.total_bytes,
s.stats_reset
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index c68c0481f42..93ed2eb368e 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -1955,10 +1955,11 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
PgStat_StatReplSlotEntry repSlotStat;
/* Nothing to do if we don't have any replication stats to be sent. */
- if (rb->spillBytes <= 0 && rb->streamBytes <= 0 && rb->totalBytes <= 0)
+ if (rb->spillBytes <= 0 && rb->streamBytes <= 0 && rb->totalBytes <= 0 &&
+ rb->memExceededCount <= 0)
return;
- elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
+ elog(DEBUG2, "UpdateDecodingStats: updating stats %p %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64 " %" PRId64,
rb,
rb->spillTxns,
rb->spillCount,
@@ -1966,6 +1967,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamTxns,
rb->streamCount,
rb->streamBytes,
+ rb->memExceededCount,
rb->totalTxns,
rb->totalBytes);
@@ -1975,6 +1977,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
repSlotStat.stream_txns = rb->streamTxns;
repSlotStat.stream_count = rb->streamCount;
repSlotStat.stream_bytes = rb->streamBytes;
+ repSlotStat.mem_exceeded_count = rb->memExceededCount;
repSlotStat.total_txns = rb->totalTxns;
repSlotStat.total_bytes = rb->totalBytes;
@@ -1986,6 +1989,7 @@ UpdateDecodingStats(LogicalDecodingContext *ctx)
rb->streamTxns = 0;
rb->streamCount = 0;
rb->streamBytes = 0;
+ rb->memExceededCount = 0;
rb->totalTxns = 0;
rb->totalBytes = 0;
}
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index a5e165fb123..a54434151de 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -390,6 +390,7 @@ ReorderBufferAllocate(void)
buffer->streamTxns = 0;
buffer->streamCount = 0;
buffer->streamBytes = 0;
+ buffer->memExceededCount = 0;
buffer->totalTxns = 0;
buffer->totalBytes = 0;
@@ -3898,14 +3899,26 @@ static void
ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
{
ReorderBufferTXN *txn;
+ bool update_stats = true;
- /*
- * Bail out if debug_logical_replication_streaming is buffered and we
- * haven't exceeded the memory limit.
- */
- if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED &&
- rb->size < logical_decoding_work_mem * (Size) 1024)
+ if (rb->size >= logical_decoding_work_mem * (Size) 1024)
+ {
+ /*
+ * Update the statistics as the memory usage has reached the limit. We
+ * report the statistics update later in this function since we can
+ * update the slot statistics altogether while streaming or
+ * serializing transactions in most cases.
+ */
+ rb->memExceededCount += 1;
+ }
+ else if (debug_logical_replication_streaming == DEBUG_LOGICAL_REP_STREAMING_BUFFERED)
+ {
+ /*
+ * Bail out if debug_logical_replication_streaming is buffered and we
+ * haven't exceeded the memory limit.
+ */
return;
+ }
/*
* If debug_logical_replication_streaming is immediate, loop until there's
@@ -3965,8 +3978,17 @@ ReorderBufferCheckMemoryLimit(ReorderBuffer *rb)
*/
Assert(txn->size == 0);
Assert(txn->nentries_mem == 0);
+
+ /*
+ * We've reported the memExceededCount update while streaming or
+ * serializing the transaction.
+ */
+ update_stats = false;
}
+ if (update_stats)
+ UpdateDecodingStats((LogicalDecodingContext *) rb->private_data);
+
/* We must be under the memory limit now. */
Assert(rb->size < logical_decoding_work_mem * (Size) 1024);
}
diff --git a/src/backend/utils/activity/pgstat_replslot.c b/src/backend/utils/activity/pgstat_replslot.c
index ccfb11c49bf..d210c261ac6 100644
--- a/src/backend/utils/activity/pgstat_replslot.c
+++ b/src/backend/utils/activity/pgstat_replslot.c
@@ -94,6 +94,7 @@ pgstat_report_replslot(ReplicationSlot *slot, const PgStat_StatReplSlotEntry *re
REPLSLOT_ACC(stream_txns);
REPLSLOT_ACC(stream_count);
REPLSLOT_ACC(stream_bytes);
+ REPLSLOT_ACC(mem_exceeded_count);
REPLSLOT_ACC(total_txns);
REPLSLOT_ACC(total_bytes);
#undef REPLSLOT_ACC
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7e89a8048d5..7e60fc62b93 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -2103,7 +2103,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
Datum
pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_REPLICATION_SLOT_COLS 10
+#define PG_STAT_GET_REPLICATION_SLOT_COLS 11
text *slotname_text = PG_GETARG_TEXT_P(0);
NameData slotname;
TupleDesc tupdesc;
@@ -2128,11 +2128,13 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
INT8OID, -1, 0);
TupleDescInitEntry(tupdesc, (AttrNumber) 7, "stream_bytes",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 8, "total_txns",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "mem_exceeded_count",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_bytes",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 9, "total_txns",
INT8OID, -1, 0);
- TupleDescInitEntry(tupdesc, (AttrNumber) 10, "stats_reset",
+ TupleDescInitEntry(tupdesc, (AttrNumber) 10, "total_bytes",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 11, "stats_reset",
TIMESTAMPTZOID, -1, 0);
BlessTupleDesc(tupdesc);
@@ -2155,13 +2157,14 @@ pg_stat_get_replication_slot(PG_FUNCTION_ARGS)
values[4] = Int64GetDatum(slotent->stream_txns);
values[5] = Int64GetDatum(slotent->stream_count);
values[6] = Int64GetDatum(slotent->stream_bytes);
- values[7] = Int64GetDatum(slotent->total_txns);
- values[8] = Int64GetDatum(slotent->total_bytes);
+ values[7] = Int64GetDatum(slotent->mem_exceeded_count);
+ values[8] = Int64GetDatum(slotent->total_txns);
+ values[9] = Int64GetDatum(slotent->total_bytes);
if (slotent->stat_reset_timestamp == 0)
- nulls[9] = true;
+ nulls[10] = true;
else
- values[9] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
+ values[10] = TimestampTzGetDatum(slotent->stat_reset_timestamp);
/* Returns the record as Datum */
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7c20180637f..3035826ef81 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5691,9 +5691,9 @@
{ oid => '6169', descr => 'statistics: information about replication slot',
proname => 'pg_stat_get_replication_slot', provolatile => 's',
proparallel => 'r', prorettype => 'record', proargtypes => 'text',
- proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,total_txns,total_bytes,stats_reset}',
+ proallargtypes => '{text,text,int8,int8,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{slot_name,slot_name,spill_txns,spill_count,spill_bytes,stream_txns,stream_count,stream_bytes,mem_exceeded_count,total_txns,total_bytes,stats_reset}',
prosrc => 'pg_stat_get_replication_slot' },
{ oid => '6230', descr => 'statistics: check if a stats object exists',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 8e8adb01176..8add724e031 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -394,6 +394,7 @@ typedef struct PgStat_StatReplSlotEntry
PgStat_Counter stream_txns;
PgStat_Counter stream_count;
PgStat_Counter stream_bytes;
+ PgStat_Counter mem_exceeded_count;
PgStat_Counter total_txns;
PgStat_Counter total_bytes;
TimestampTz stat_reset_timestamp;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 91dc7e5e448..3cbe106a3c7 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -690,6 +690,9 @@ struct ReorderBuffer
int64 streamCount; /* streaming invocation counter */
int64 streamBytes; /* amount of data decoded */
+ /* Number of times the logical_decoding_work_mem limit has been reached */
+ int64 memExceededCount;
+
/*
* Statistics about all the transactions sent to the decoding output
* plugin
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 7f1cb3bb4af..35b792d4739 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2140,11 +2140,12 @@ pg_stat_replication_slots| SELECT s.slot_name,
s.stream_txns,
s.stream_count,
s.stream_bytes,
+ s.mem_exceeded_count,
s.total_txns,
s.total_bytes,
s.stats_reset
FROM pg_replication_slots r,
- LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, total_txns, total_bytes, stats_reset)
+ LATERAL pg_stat_get_replication_slot((r.slot_name)::text) s(slot_name, spill_txns, spill_count, spill_bytes, stream_txns, stream_count, stream_bytes, mem_exceeded_count, total_txns, total_bytes, stats_reset)
WHERE (r.datoid IS NOT NULL);
pg_stat_slru| SELECT name,
blks_zeroed,
--
2.47.3
Hi,
On Tue, Oct 07, 2025 at 10:44:33AM -0700, Masahiko Sawada wrote:
Thank you for the comment. I've noted this discussion as a comment in
the new tests.
Thanks!
I've attached the updated version patch. Please review it.
LGTM.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
On Tue, Oct 7, 2025 at 10:02 PM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
Hi,
On Tue, Oct 07, 2025 at 10:44:33AM -0700, Masahiko Sawada wrote:
Thank you for the comment. I've noted this discussion as a comment in
the new tests.Thanks!
I've attached the updated version patch. Please review it.
LGTM.
Thank you! Pushed.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com