Improve explicit cursor handling in pg_stat_statements
Hi hackers,
I recently looked into a workload that makes heavy use of explicit cursors,
and I found that pg_stat_statements can become a bottleneck. The
application in question declares hundreds of cursors, and for each one,
performs many FETCH and MOVE operations with varying fetch sizes.
As a result, pg_stat_statements ends up overwhelmed by the deallocation
(and garbage collection) of DECLARE CURSOR, FETCH, and MOVE entries.
Each of these is associated with a unique queryId, which leads to bloated
entries with limited diagnostic value.
Other issues:
1. FETCH/MOVE statements don't really help much in laying blame on a specific
query. the DECLARE CURSOR statement could have been evicted in
pg_stat_statements
by that point or a similar cursor name is pointing to a different query.
Also, FETCH doesn't aggregate for the same cursor — e.g., FETCH 10 c1 and
FETCH 20 c1 show up as separate entries.
2. DECLARE CURSOR doesn't provide execution stats for the underlying SQL.
Enabling pg_stat_statements.track = 'all' can expose the underlying SQL,
but adds overhead.There’s also a bug: the toplevel column for the underlying
query is still marked as "t", even though you must set track "all" to see it.
Based on this, I propose the following improvements:
1. Better normalization of cursor utility commands:
2. Normalize the cursor name in CLOSE.
3. Normalize fetch/move sizes in FETCH and MOVE. Users can use the rows and
calls columns to derive average fetch size. Ideally I would want to
normalize the
cursor name and generate the queryId f the FETCH statement based on the
underlying query, but that is not possible to do that post parsing.
(The above normalizations of these utility statements will reduce the bloat.)
4. Track the underlying query of a cursor by default, even when
pg_stat_statements.track_utility = off.
I’ve attached two patches that implement this. Here's a quick example:
```
begin;
declare cs1 cursor for select from pg_class;
declare cs2 cursor for select from pg_class;
fetch 10 cs1;
fetch 20 cs1;
fetch 10 cs1;
fetch 10 cs2;
close cs1;
close cs2;
declare cs1 cursor for select from pg_attribute;
SELECT calls, rows, query, toplevel FROM pg_stat_statements ORDER BY
query COLLATE "C";
commit;
```
current state:
```
postgres=*# SELECT calls, rows, query, toplevel FROM
pg_stat_statements ORDER BY query COLLATE "C";
calls | rows |
query | toplevel
-------+------+-----------------------------------------------------------------------------------------------------------+----------
1 | 1 | SELECT name FROM pg_catalog.pg_available_extensions
WHERE name LIKE $1 AND installed_version IS NULL+| t
| | LIMIT $2
|
1 | 0 | begin
| t
1 | 0 | close cs1
| t
1 | 0 | close cs2
| t
1 | 0 | create extension pg_stat_statements
| t
1 | 0 | declare cs1 cursor for select from pg_attribute
| t
1 | 0 | declare cs1 cursor for select from pg_class
| t
1 | 0 | declare cs2 cursor for select from pg_class
| t
2 | 20 | fetch 10 cs1
| t
1 | 10 | fetch 10 cs2
| t
1 | 20 | fetch 20 cs1
| t
(11 rows)
```
with both patches applied:
```
postgres=*# SELECT calls, rows, query, toplevel FROM
pg_stat_statements ORDER BY query COLLATE "C";
calls | rows | query | toplevel
-------+------+------------------------------------------------+----------
1 | 0 | begin | t
2 | 0 | close $1 | t
1 | 0 | declare $1 cursor for select from pg_attribute | t
2 | 0 | declare $1 cursor for select from pg_class | t
3 | 40 | fetch $1 cs1 | t
1 | 10 | fetch $1 cs2 | t
2 | 50 | select from pg_class | t
(7 rows)
postgres=*# commit;
COMMIT
```
FWIW, I raised this ~3 years ago [0]/messages/by-id/203CFCF7-176E-4AFC-A48E-B2CECFECD6AA@amazon.com, but there was not much interest. I
have seen this being a problem a few times since then that I think something
should be done about. I also was not happy with the approach I took in [0]/messages/by-id/203CFCF7-176E-4AFC-A48E-B2CECFECD6AA@amazon.com.
Looking forward to feedback!
Regards,
--
Sami Imseih
Amazon Web Services (AWS)
[0]: /messages/by-id/203CFCF7-176E-4AFC-A48E-B2CECFECD6AA@amazon.com
Attachments:
v1-0001-Improve-cursor-handling-in-pg_stat_statements.patchapplication/octet-stream; name=v1-0001-Improve-cursor-handling-in-pg_stat_statements.patchDownload
From 0176ea9412c4ad49621fb7e3267eed403d9fe6fb Mon Sep 17 00:00:00 2001
From: Sami Imseih <simseih@amazon.com>
Date: Wed, 30 Apr 2025 10:13:15 -0500
Subject: [PATCH v1 1/1] Improve cursor handling in pg_stat_statements
---
.../pg_stat_statements/expected/cursors.out | 6 ++-
.../expected/level_tracking.out | 5 +-
.../pg_stat_statements/expected/utility.out | 3 +-
.../pg_stat_statements/pg_stat_statements.c | 54 +++++++++++++------
4 files changed, 46 insertions(+), 22 deletions(-)
diff --git a/contrib/pg_stat_statements/expected/cursors.out b/contrib/pg_stat_statements/expected/cursors.out
index 0fc4b2c098d..b7bda01d8f1 100644
--- a/contrib/pg_stat_statements/expected/cursors.out
+++ b/contrib/pg_stat_statements/expected/cursors.out
@@ -20,8 +20,9 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
-------+------+-------------------------------------------------------
2 | 0 | CLOSE cursor_stats_1
2 | 0 | DECLARE cursor_stats_1 CURSOR WITH HOLD FOR SELECT $1
+ 2 | 2 | SELECT $1
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(3 rows)
+(4 rows)
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
t
@@ -59,8 +60,9 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 0 | DECLARE cursor_stats_2 CURSOR WITH HOLD FOR SELECT $1
1 | 1 | FETCH 1 IN cursor_stats_1
1 | 1 | FETCH 1 IN cursor_stats_2
+ 2 | 2 | SELECT $1
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(9 rows)
+(10 rows)
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
t
diff --git a/contrib/pg_stat_statements/expected/level_tracking.out b/contrib/pg_stat_statements/expected/level_tracking.out
index 03bea14d5da..bba78289962 100644
--- a/contrib/pg_stat_statements/expected/level_tracking.out
+++ b/contrib/pg_stat_statements/expected/level_tracking.out
@@ -1143,7 +1143,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * from stats_track_tab
t | 1 | FETCH FORWARD 1 FROM foocur
- f | 1 | SELECT * from stats_track_tab
+ t | 1 | SELECT * from stats_track_tab
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(7 rows)
@@ -1173,8 +1173,9 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * FROM stats_track_tab
t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | SELECT * FROM stats_track_tab
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(6 rows)
+(7 rows)
-- COPY - all-level tracking.
SET pg_stat_statements.track = 'all';
diff --git a/contrib/pg_stat_statements/expected/utility.out b/contrib/pg_stat_statements/expected/utility.out
index aa4f0f7e628..c19454754dc 100644
--- a/contrib/pg_stat_statements/expected/utility.out
+++ b/contrib/pg_stat_statements/expected/utility.out
@@ -706,9 +706,10 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 7 | FETCH FORWARD ALL pgss_cursor
1 | 1 | FETCH NEXT pgss_cursor
1 | 13 | REFRESH MATERIALIZED VIEW pgss_matv
+ 1 | 13 | SELECT * FROM pgss_matv
1 | 10 | SELECT generate_series($1, $2) c INTO pgss_select_into
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(12 rows)
+(13 rows)
DROP MATERIALIZED VIEW pgss_matv;
DROP TABLE pgss_ctas;
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 9778407cba3..91d26518512 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -311,6 +311,11 @@ static bool pgss_save = true; /* whether to save stats across shutdown */
SpinLockRelease(&pgss->mutex); \
} while(0)
+#define is_cursor_utility() \
+ (IsA(parsetree, DeclareCursorStmt) || \
+ IsA(parsetree, FetchStmt) || \
+ IsA(parsetree, ClosePortalStmt))
+
/*---- Function declarations ----*/
PG_FUNCTION_INFO_V1(pg_stat_statements_reset);
@@ -1123,6 +1128,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
int saved_stmt_location = pstmt->stmt_location;
int saved_stmt_len = pstmt->stmt_len;
bool enabled = pgss_track_utility && pgss_enabled(nesting_level);
+ bool bump_level;
/*
* Force utility statements to get queryId zero. We do this even in cases
@@ -1153,8 +1159,14 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
* ensuing EXECUTEs. This would be confusing. Since PREPARE doesn't
* actually run the planner (only parse+rewrite), its costs are generally
* pretty negligible and it seems okay to just ignore it.
+ *
+ * If it's a cursor related utility command, don't bump the level and
+ * force the tracking of the underlying sql.
*/
- if (enabled &&
+ bump_level =
+ !is_cursor_utility();
+
+ if ((enabled || !bump_level) &&
!IsA(parsetree, ExecuteStmt) &&
!IsA(parsetree, PrepareStmt))
{
@@ -1165,12 +1177,14 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
bufusage;
WalUsage walusage_start,
walusage;
+ bool store_utility = true;
bufusage_start = pgBufferUsage;
walusage_start = pgWalUsage;
INSTR_TIME_SET_CURRENT(start);
- nesting_level++;
+ if (bump_level)
+ nesting_level++;
PG_TRY();
{
if (prev_ProcessUtility)
@@ -1184,7 +1198,8 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
}
PG_FINALLY();
{
- nesting_level--;
+ if (bump_level)
+ nesting_level--;
}
PG_END_TRY();
@@ -1220,19 +1235,24 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
memset(&walusage, 0, sizeof(WalUsage));
WalUsageAccumDiff(&walusage, &pgWalUsage, &walusage_start);
- pgss_store(queryString,
- saved_queryId,
- saved_stmt_location,
- saved_stmt_len,
- PGSS_EXEC,
- INSTR_TIME_GET_MILLISEC(duration),
- rows,
- &bufusage,
- &walusage,
- NULL,
- NULL,
- 0,
- 0);
+ /*
+ * In case we got here because of a cursor command, let's make sure we
+ * have utility tracking enabled, again.
+ */
+ if (enabled)
+ pgss_store(queryString,
+ saved_queryId,
+ saved_stmt_location,
+ saved_stmt_len,
+ PGSS_EXEC,
+ INSTR_TIME_GET_MILLISEC(duration),
+ rows,
+ &bufusage,
+ &walusage,
+ NULL,
+ NULL,
+ 0,
+ 0);
}
else
{
@@ -1248,7 +1268,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
* To be absolutely certain we don't mess up the nesting level,
* evaluate the bump_level condition just once.
*/
- bool bump_level =
+ bump_level =
!IsA(parsetree, ExecuteStmt) &&
!IsA(parsetree, PrepareStmt);
--
2.39.5 (Apple Git-154)
v1-0001-Normalize-cursor-utility-statements.patchapplication/octet-stream; name=v1-0001-Normalize-cursor-utility-statements.patchDownload
From 9690c6dfefa60c61cc99691210c31f5bcbfbe706 Mon Sep 17 00:00:00 2001
From: Sami Imseih <simseih@amazon.com>
Date: Wed, 30 Apr 2025 11:21:30 -0500
Subject: [PATCH v1 1/1] Normalize cursor utility statements
---
src/backend/parser/gram.y | 18 ++++++++++++++++++
src/include/nodes/parsenodes.h | 12 +++++++++---
2 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 3c4268b271a..c9be19643ac 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -3341,6 +3341,7 @@ ClosePortalStmt:
ClosePortalStmt *n = makeNode(ClosePortalStmt);
n->portalname = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| CLOSE ALL
@@ -3348,6 +3349,7 @@ ClosePortalStmt:
ClosePortalStmt *n = makeNode(ClosePortalStmt);
n->portalname = NULL;
+ n->location = -1;
$$ = (Node *) n;
}
;
@@ -7479,6 +7481,7 @@ fetch_args: cursor_name
n->portalname = $1;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| from_in cursor_name
@@ -7488,6 +7491,7 @@ fetch_args: cursor_name
n->portalname = $2;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| NEXT opt_from_in cursor_name
@@ -7497,6 +7501,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| PRIOR opt_from_in cursor_name
@@ -7506,6 +7511,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| FIRST_P opt_from_in cursor_name
@@ -7515,6 +7521,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| LAST_P opt_from_in cursor_name
@@ -7524,6 +7531,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = -1;
+ n->location = -1;
$$ = (Node *) n;
}
| ABSOLUTE_P SignedIconst opt_from_in cursor_name
@@ -7533,6 +7541,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_ABSOLUTE;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| RELATIVE_P SignedIconst opt_from_in cursor_name
@@ -7542,6 +7551,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_RELATIVE;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| SignedIconst opt_from_in cursor_name
@@ -7551,6 +7561,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = $1;
+ n->location = @1;
$$ = (Node *) n;
}
| ALL opt_from_in cursor_name
@@ -7560,6 +7571,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
$$ = (Node *) n;
}
| FORWARD opt_from_in cursor_name
@@ -7569,6 +7581,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| FORWARD SignedIconst opt_from_in cursor_name
@@ -7578,6 +7591,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| FORWARD ALL opt_from_in cursor_name
@@ -7596,6 +7610,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| BACKWARD SignedIconst opt_from_in cursor_name
@@ -7605,6 +7620,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| BACKWARD ALL opt_from_in cursor_name
@@ -7614,6 +7630,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
$$ = (Node *) n;
}
;
@@ -12752,6 +12769,7 @@ DeclareCursorStmt: DECLARE cursor_name cursor_options CURSOR opt_hold FOR Select
DeclareCursorStmt *n = makeNode(DeclareCursorStmt);
n->portalname = $2;
+ n->location = @2;
/* currently we always set FAST_PLAN option */
n->options = $3 | $5 | CURSOR_OPT_FAST_PLAN;
n->query = $7;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4610fc61293..14181a48924 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3387,9 +3387,11 @@ typedef struct SecLabelStmt
typedef struct DeclareCursorStmt
{
NodeTag type;
- char *portalname; /* name of the portal (cursor) */
+ /* name of the portal (cursor) */
+ char *portalname pg_node_attr(query_jumble_ignore);
int options; /* bitmask of options (see above) */
Node *query; /* the query (see comments above) */
+ ParseLoc location pg_node_attr(query_jumble_location);
} DeclareCursorStmt;
/* ----------------------
@@ -3399,7 +3401,9 @@ typedef struct DeclareCursorStmt
typedef struct ClosePortalStmt
{
NodeTag type;
- char *portalname; /* name of the portal (cursor) */
+ /* name of the portal (cursor) */
+ char *portalname pg_node_attr(query_jumble_ignore);
+ ParseLoc location pg_node_attr(query_jumble_location);
/* NULL means CLOSE ALL */
} ClosePortalStmt;
@@ -3423,9 +3427,11 @@ typedef struct FetchStmt
{
NodeTag type;
FetchDirection direction; /* see above */
- long howMany; /* number of rows, or position argument */
+ /* number of rows, or position argument */
+ long howMany pg_node_attr(query_jumble_ignore);
char *portalname; /* name of portal (cursor) */
bool ismove; /* true if MOVE */
+ ParseLoc location pg_node_attr(query_jumble_location);
} FetchStmt;
/* ----------------------
--
2.39.5 (Apple Git-154)
I forgot to add the proper tests to the Normalize cursor utility statements.
Reattaching v2.
I also want to add that the decision to not normalize the cursor name in
the FETCH command is because it would not make sense to combine
FETCH commands for various cursors into the same entry.
Regards,
--
Sami Imseih
Amazon Web Services (AWS)
Attachments:
v2-0002-Improve-cursor-handling-in-pg_stat_statements.patchapplication/octet-stream; name=v2-0002-Improve-cursor-handling-in-pg_stat_statements.patchDownload
From 1746dff6639fadcbd0f530144483ccb83e44123f Mon Sep 17 00:00:00 2001
From: Sami Imseih <simseih@amazon.com>
Date: Wed, 30 Apr 2025 14:27:39 -0500
Subject: [PATCH v2 2/2] Improve cursor handling in pg_stat_statements
---
.../pg_stat_statements/expected/cursors.out | 6 ++-
.../expected/level_tracking.out | 5 +-
.../pg_stat_statements/expected/utility.out | 3 +-
.../pg_stat_statements/pg_stat_statements.c | 54 +++++++++++++------
4 files changed, 46 insertions(+), 22 deletions(-)
diff --git a/contrib/pg_stat_statements/expected/cursors.out b/contrib/pg_stat_statements/expected/cursors.out
index 1f4de404707..5c4103e33d6 100644
--- a/contrib/pg_stat_statements/expected/cursors.out
+++ b/contrib/pg_stat_statements/expected/cursors.out
@@ -20,8 +20,9 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
-------+------+----------------------------------------------------
2 | 0 | CLOSE $1
2 | 0 | DECLARE $1 CURSOR WITH HOLD FOR SELECT $2
+ 2 | 2 | SELECT $1
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(3 rows)
+(4 rows)
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
t
@@ -57,8 +58,9 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
2 | 0 | DECLARE $1 CURSOR WITH HOLD FOR SELECT $2
1 | 1 | FETCH $1 IN cursor_stats_1
1 | 1 | FETCH $1 IN cursor_stats_2
+ 2 | 2 | SELECT $1
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(7 rows)
+(8 rows)
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
t
diff --git a/contrib/pg_stat_statements/expected/level_tracking.out b/contrib/pg_stat_statements/expected/level_tracking.out
index 2b2f1d5ee42..e587c3269ac 100644
--- a/contrib/pg_stat_statements/expected/level_tracking.out
+++ b/contrib/pg_stat_statements/expected/level_tracking.out
@@ -1143,7 +1143,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | COMMIT
t | 1 | DECLARE $1 CURSOR FOR SELECT * from stats_track_tab
t | 1 | FETCH FORWARD $1 FROM foocur
- f | 1 | SELECT * from stats_track_tab
+ t | 1 | SELECT * from stats_track_tab
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(7 rows)
@@ -1173,8 +1173,9 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | COMMIT
t | 1 | DECLARE $1 CURSOR FOR SELECT * FROM stats_track_tab
t | 1 | FETCH FORWARD $1 FROM foocur
+ t | 1 | SELECT * FROM stats_track_tab
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(6 rows)
+(7 rows)
-- COPY - all-level tracking.
SET pg_stat_statements.track = 'all';
diff --git a/contrib/pg_stat_statements/expected/utility.out b/contrib/pg_stat_statements/expected/utility.out
index e8b27d8304b..bb91e97941a 100644
--- a/contrib/pg_stat_statements/expected/utility.out
+++ b/contrib/pg_stat_statements/expected/utility.out
@@ -704,9 +704,10 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 0 | DECLARE $1 CURSOR FOR SELECT * FROM pgss_matv
3 | 13 | FETCH NEXT pgss_cursor
1 | 13 | REFRESH MATERIALIZED VIEW pgss_matv
+ 1 | 13 | SELECT * FROM pgss_matv
1 | 10 | SELECT generate_series($1, $2) c INTO pgss_select_into
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(10 rows)
+(11 rows)
DROP MATERIALIZED VIEW pgss_matv;
DROP TABLE pgss_ctas;
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 9778407cba3..91d26518512 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -311,6 +311,11 @@ static bool pgss_save = true; /* whether to save stats across shutdown */
SpinLockRelease(&pgss->mutex); \
} while(0)
+#define is_cursor_utility() \
+ (IsA(parsetree, DeclareCursorStmt) || \
+ IsA(parsetree, FetchStmt) || \
+ IsA(parsetree, ClosePortalStmt))
+
/*---- Function declarations ----*/
PG_FUNCTION_INFO_V1(pg_stat_statements_reset);
@@ -1123,6 +1128,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
int saved_stmt_location = pstmt->stmt_location;
int saved_stmt_len = pstmt->stmt_len;
bool enabled = pgss_track_utility && pgss_enabled(nesting_level);
+ bool bump_level;
/*
* Force utility statements to get queryId zero. We do this even in cases
@@ -1153,8 +1159,14 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
* ensuing EXECUTEs. This would be confusing. Since PREPARE doesn't
* actually run the planner (only parse+rewrite), its costs are generally
* pretty negligible and it seems okay to just ignore it.
+ *
+ * If it's a cursor related utility command, don't bump the level and
+ * force the tracking of the underlying sql.
*/
- if (enabled &&
+ bump_level =
+ !is_cursor_utility();
+
+ if ((enabled || !bump_level) &&
!IsA(parsetree, ExecuteStmt) &&
!IsA(parsetree, PrepareStmt))
{
@@ -1165,12 +1177,14 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
bufusage;
WalUsage walusage_start,
walusage;
+ bool store_utility = true;
bufusage_start = pgBufferUsage;
walusage_start = pgWalUsage;
INSTR_TIME_SET_CURRENT(start);
- nesting_level++;
+ if (bump_level)
+ nesting_level++;
PG_TRY();
{
if (prev_ProcessUtility)
@@ -1184,7 +1198,8 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
}
PG_FINALLY();
{
- nesting_level--;
+ if (bump_level)
+ nesting_level--;
}
PG_END_TRY();
@@ -1220,19 +1235,24 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
memset(&walusage, 0, sizeof(WalUsage));
WalUsageAccumDiff(&walusage, &pgWalUsage, &walusage_start);
- pgss_store(queryString,
- saved_queryId,
- saved_stmt_location,
- saved_stmt_len,
- PGSS_EXEC,
- INSTR_TIME_GET_MILLISEC(duration),
- rows,
- &bufusage,
- &walusage,
- NULL,
- NULL,
- 0,
- 0);
+ /*
+ * In case we got here because of a cursor command, let's make sure we
+ * have utility tracking enabled, again.
+ */
+ if (enabled)
+ pgss_store(queryString,
+ saved_queryId,
+ saved_stmt_location,
+ saved_stmt_len,
+ PGSS_EXEC,
+ INSTR_TIME_GET_MILLISEC(duration),
+ rows,
+ &bufusage,
+ &walusage,
+ NULL,
+ NULL,
+ 0,
+ 0);
}
else
{
@@ -1248,7 +1268,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
* To be absolutely certain we don't mess up the nesting level,
* evaluate the bump_level condition just once.
*/
- bool bump_level =
+ bump_level =
!IsA(parsetree, ExecuteStmt) &&
!IsA(parsetree, PrepareStmt);
--
2.39.5 (Apple Git-154)
v2-0001-Normalize-cursor-utility-statements.patchapplication/octet-stream; name=v2-0001-Normalize-cursor-utility-statements.patchDownload
From ddeb645a00084335e42ea13b1c91406160b749fd Mon Sep 17 00:00:00 2001
From: Sami Imseih <simseih@amazon.com>
Date: Wed, 30 Apr 2025 14:25:28 -0500
Subject: [PATCH v2 1/2] Normalize cursor utility statements
---
.../pg_stat_statements/expected/cursors.out | 24 +++++++++----------
.../expected/level_tracking.out | 24 +++++++++----------
.../pg_stat_statements/expected/utility.out | 8 +++----
src/backend/parser/gram.y | 18 ++++++++++++++
src/include/nodes/parsenodes.h | 12 +++++++---
5 files changed, 53 insertions(+), 33 deletions(-)
diff --git a/contrib/pg_stat_statements/expected/cursors.out b/contrib/pg_stat_statements/expected/cursors.out
index 0fc4b2c098d..1f4de404707 100644
--- a/contrib/pg_stat_statements/expected/cursors.out
+++ b/contrib/pg_stat_statements/expected/cursors.out
@@ -16,10 +16,10 @@ CLOSE cursor_stats_1;
DECLARE cursor_stats_1 CURSOR WITH HOLD FOR SELECT 2;
CLOSE cursor_stats_1;
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
- calls | rows | query
--------+------+-------------------------------------------------------
- 2 | 0 | CLOSE cursor_stats_1
- 2 | 0 | DECLARE cursor_stats_1 CURSOR WITH HOLD FOR SELECT $1
+ calls | rows | query
+-------+------+----------------------------------------------------
+ 2 | 0 | CLOSE $1
+ 2 | 0 | DECLARE $1 CURSOR WITH HOLD FOR SELECT $2
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(3 rows)
@@ -49,18 +49,16 @@ CLOSE cursor_stats_1;
CLOSE cursor_stats_2;
COMMIT;
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
- calls | rows | query
--------+------+-------------------------------------------------------
+ calls | rows | query
+-------+------+----------------------------------------------------
1 | 0 | BEGIN
- 1 | 0 | CLOSE cursor_stats_1
- 1 | 0 | CLOSE cursor_stats_2
+ 2 | 0 | CLOSE $1
1 | 0 | COMMIT
- 1 | 0 | DECLARE cursor_stats_1 CURSOR WITH HOLD FOR SELECT $1
- 1 | 0 | DECLARE cursor_stats_2 CURSOR WITH HOLD FOR SELECT $1
- 1 | 1 | FETCH 1 IN cursor_stats_1
- 1 | 1 | FETCH 1 IN cursor_stats_2
+ 2 | 0 | DECLARE $1 CURSOR WITH HOLD FOR SELECT $2
+ 1 | 1 | FETCH $1 IN cursor_stats_1
+ 1 | 1 | FETCH $1 IN cursor_stats_2
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(9 rows)
+(7 rows)
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
t
diff --git a/contrib/pg_stat_statements/expected/level_tracking.out b/contrib/pg_stat_statements/expected/level_tracking.out
index 03bea14d5da..2b2f1d5ee42 100644
--- a/contrib/pg_stat_statements/expected/level_tracking.out
+++ b/contrib/pg_stat_statements/expected/level_tracking.out
@@ -921,7 +921,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
toplevel | calls | query
----------+-------+------------------------------------------------------------------------------
t | 1 | EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF) +
- | | DECLARE foocur CURSOR FOR SELECT * FROM stats_track_tab
+ | | DECLARE $1 CURSOR FOR SELECT * FROM stats_track_tab
t | 1 | EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF) SELECT $1
f | 1 | SELECT $1
f | 1 | SELECT * FROM stats_track_tab
@@ -954,7 +954,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
toplevel | calls | query
----------+-------+------------------------------------------------------------------------------
t | 1 | EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF) +
- | | DECLARE foocur CURSOR FOR SELECT * FROM stats_track_tab
+ | | DECLARE $1 CURSOR FOR SELECT * FROM stats_track_tab
t | 1 | EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF) SELECT $1
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(3 rows)
@@ -1136,13 +1136,13 @@ CLOSE foocur;
COMMIT;
SELECT toplevel, calls, query FROM pg_stat_statements
ORDER BY query COLLATE "C";
- toplevel | calls | query
-----------+-------+---------------------------------------------------------
+ toplevel | calls | query
+----------+-------+-----------------------------------------------------
t | 1 | BEGIN
- t | 1 | CLOSE foocur
+ t | 1 | CLOSE $1
t | 1 | COMMIT
- t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * from stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | DECLARE $1 CURSOR FOR SELECT * from stats_track_tab
+ t | 1 | FETCH FORWARD $1 FROM foocur
f | 1 | SELECT * from stats_track_tab
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(7 rows)
@@ -1166,13 +1166,13 @@ CLOSE foocur;
COMMIT;
SELECT toplevel, calls, query FROM pg_stat_statements
ORDER BY query COLLATE "C";
- toplevel | calls | query
-----------+-------+---------------------------------------------------------
+ toplevel | calls | query
+----------+-------+-----------------------------------------------------
t | 1 | BEGIN
- t | 1 | CLOSE foocur
+ t | 1 | CLOSE $1
t | 1 | COMMIT
- t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * FROM stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | DECLARE $1 CURSOR FOR SELECT * FROM stats_track_tab
+ t | 1 | FETCH FORWARD $1 FROM foocur
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(6 rows)
diff --git a/contrib/pg_stat_statements/expected/utility.out b/contrib/pg_stat_statements/expected/utility.out
index aa4f0f7e628..e8b27d8304b 100644
--- a/contrib/pg_stat_statements/expected/utility.out
+++ b/contrib/pg_stat_statements/expected/utility.out
@@ -701,14 +701,12 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 3 | COPY pgss_ctas (a, b) FROM STDIN
1 | 13 | CREATE MATERIALIZED VIEW pgss_matv AS SELECT * FROM pgss_ctas
1 | 10 | CREATE TABLE pgss_ctas AS SELECT a, $1 b FROM generate_series($2, $3) a
- 1 | 0 | DECLARE pgss_cursor CURSOR FOR SELECT * FROM pgss_matv
- 1 | 5 | FETCH FORWARD 5 pgss_cursor
- 1 | 7 | FETCH FORWARD ALL pgss_cursor
- 1 | 1 | FETCH NEXT pgss_cursor
+ 1 | 0 | DECLARE $1 CURSOR FOR SELECT * FROM pgss_matv
+ 3 | 13 | FETCH NEXT pgss_cursor
1 | 13 | REFRESH MATERIALIZED VIEW pgss_matv
1 | 10 | SELECT generate_series($1, $2) c INTO pgss_select_into
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(12 rows)
+(10 rows)
DROP MATERIALIZED VIEW pgss_matv;
DROP TABLE pgss_ctas;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 3c4268b271a..c9be19643ac 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -3341,6 +3341,7 @@ ClosePortalStmt:
ClosePortalStmt *n = makeNode(ClosePortalStmt);
n->portalname = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| CLOSE ALL
@@ -3348,6 +3349,7 @@ ClosePortalStmt:
ClosePortalStmt *n = makeNode(ClosePortalStmt);
n->portalname = NULL;
+ n->location = -1;
$$ = (Node *) n;
}
;
@@ -7479,6 +7481,7 @@ fetch_args: cursor_name
n->portalname = $1;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| from_in cursor_name
@@ -7488,6 +7491,7 @@ fetch_args: cursor_name
n->portalname = $2;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| NEXT opt_from_in cursor_name
@@ -7497,6 +7501,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| PRIOR opt_from_in cursor_name
@@ -7506,6 +7511,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| FIRST_P opt_from_in cursor_name
@@ -7515,6 +7521,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| LAST_P opt_from_in cursor_name
@@ -7524,6 +7531,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = -1;
+ n->location = -1;
$$ = (Node *) n;
}
| ABSOLUTE_P SignedIconst opt_from_in cursor_name
@@ -7533,6 +7541,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_ABSOLUTE;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| RELATIVE_P SignedIconst opt_from_in cursor_name
@@ -7542,6 +7551,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_RELATIVE;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| SignedIconst opt_from_in cursor_name
@@ -7551,6 +7561,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = $1;
+ n->location = @1;
$$ = (Node *) n;
}
| ALL opt_from_in cursor_name
@@ -7560,6 +7571,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
$$ = (Node *) n;
}
| FORWARD opt_from_in cursor_name
@@ -7569,6 +7581,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| FORWARD SignedIconst opt_from_in cursor_name
@@ -7578,6 +7591,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| FORWARD ALL opt_from_in cursor_name
@@ -7596,6 +7610,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| BACKWARD SignedIconst opt_from_in cursor_name
@@ -7605,6 +7620,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| BACKWARD ALL opt_from_in cursor_name
@@ -7614,6 +7630,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
$$ = (Node *) n;
}
;
@@ -12752,6 +12769,7 @@ DeclareCursorStmt: DECLARE cursor_name cursor_options CURSOR opt_hold FOR Select
DeclareCursorStmt *n = makeNode(DeclareCursorStmt);
n->portalname = $2;
+ n->location = @2;
/* currently we always set FAST_PLAN option */
n->options = $3 | $5 | CURSOR_OPT_FAST_PLAN;
n->query = $7;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4610fc61293..14181a48924 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3387,9 +3387,11 @@ typedef struct SecLabelStmt
typedef struct DeclareCursorStmt
{
NodeTag type;
- char *portalname; /* name of the portal (cursor) */
+ /* name of the portal (cursor) */
+ char *portalname pg_node_attr(query_jumble_ignore);
int options; /* bitmask of options (see above) */
Node *query; /* the query (see comments above) */
+ ParseLoc location pg_node_attr(query_jumble_location);
} DeclareCursorStmt;
/* ----------------------
@@ -3399,7 +3401,9 @@ typedef struct DeclareCursorStmt
typedef struct ClosePortalStmt
{
NodeTag type;
- char *portalname; /* name of the portal (cursor) */
+ /* name of the portal (cursor) */
+ char *portalname pg_node_attr(query_jumble_ignore);
+ ParseLoc location pg_node_attr(query_jumble_location);
/* NULL means CLOSE ALL */
} ClosePortalStmt;
@@ -3423,9 +3427,11 @@ typedef struct FetchStmt
{
NodeTag type;
FetchDirection direction; /* see above */
- long howMany; /* number of rows, or position argument */
+ /* number of rows, or position argument */
+ long howMany pg_node_attr(query_jumble_ignore);
char *portalname; /* name of portal (cursor) */
bool ismove; /* true if MOVE */
+ ParseLoc location pg_node_attr(query_jumble_location);
} FetchStmt;
/* ----------------------
--
2.39.5 (Apple Git-154)
On Wed, Apr 30, 2025 at 02:43:41PM -0500, Sami Imseih wrote:
I also want to add that the decision to not normalize the cursor name in
the FETCH command is because it would not make sense to combine
FETCH commands for various cursors into the same entry.
- calls | rows | query
--------+------+-------------------------------------------------------
- 2 | 0 | CLOSE cursor_stats_1
- 2 | 0 | DECLARE cursor_stats_1 CURSOR WITH HOLD FOR SELECT $1
+ calls | rows | query
+-------+------+----------------------------------------------------
+ 2 | 0 | CLOSE $1
+ 2 | 0 | DECLARE $1 CURSOR WITH HOLD FOR SELECT $2
Hmm. What are the workloads that you have seen as problematic? Do
these involve cursor names generated randomly, where most of them are
similar with a random factor for the name? Too much normalization
here would limit the amount of verbosity that we have for this area,
especially if we are dealing with query patterns that rely on few
CLOSE naming patterns spread across a couple of key queries, because
we would now know anymore about their distribution.
- 1 | 5 | FETCH FORWARD 5 pgss_cursor
- 1 | 7 | FETCH FORWARD ALL pgss_cursor
- 1 | 1 | FETCH NEXT pgss_cursor
+ 1 | 0 | DECLARE $1 CURSOR FOR SELECT * FROM pgss_matv
Saying that, applying normalization for the number of FETCH looks like
a natural move. It seems to me that we should still make a difference
with the ALL case, though?
typedef struct ClosePortalStmt
{
NodeTag type;
- char *portalname; /* name of the portal (cursor) */
+ /* name of the portal (cursor) */
+ char *portalname pg_node_attr(query_jumble_ignore);
+ ParseLoc location pg_node_attr(query_jumble_location);
/* NULL means CLOSE ALL */
Could it matter to make a distinction with CLOSE ALL, compared to the
case where the CLOSE statements are named? It would be possible to
make a difference compared to the named case with an extra boolean
field, for example. I would suggest to add some tests for CLOSE ALL
anyway; we don't have any currently.
--
Michael
Hmm. What are the workloads that you have seen as problematic? Do
these involve cursor names generated randomly, where most of them are
similar with a random factor for the name?
postgres_fdw, as an example, in which cursor name get reused
for different queries. Notice below "c1" and "c2" is reused for different
queries, so now what underlying sql is FETCH, i.e. FETCH 100 FROM c1 referring
to? v2-0001 does not help us with the FETCH problem
because as I mentioned we don't have access to the underlying sql
( and parsing is even too early to do a portal lookup to find the
underlying sql to
base the queryId on). What v2-0001 will do is at least group the DECLARE CURSOR
statements together for cursors referencing the same query and reduce the #
of entries.
```
create foreign table t2(id int) server r1;
create foreign table t1(id int) server r1;
postgres=# select * from t2, t ;
id | id
----+----
1 | 1
(1 row)
postgres=# select * from t, t2 ;
id | id
----+----
1 | 1
(1 row)
```
on the remote side
```
postgres=# select calls, query from pg_stat_statements where query like '% c%';
calls | query
-------+-----------------------------------------------------------------
1 | DECLARE c2 CURSOR FOR +
| SELECT id FROM public.t2
2 | DECLARE c1 CURSOR FOR +
| SELECT id FROM public.t2
3 | CLOSE c2
3 | CLOSE c1
2 | DECLARE c2 CURSOR FOR +
| SELECT id FROM public.t
3 | FETCH 100 FROM c1
3 | FETCH 100 FROM c2
1 | DECLARE c1 CURSOR FOR +
| SELECT id FROM public.t
2 | select calls, query from pg_stat_statements where query like $1
(9 rows)
```
Too much normalization
here would limit the amount of verbosity that we have for this area,
especially if we are dealing with query patterns that rely on few
CLOSE naming patterns spread across a couple of key queries, because
we would now know anymore about their distribution.
The FETCH and CLOSE are already not clear to what underlying SQL
they are referring to, and there is not much chance to actually
improve that unless
we track a cursor queryId in pg_stat_statements ( at that point we can show that
FETCH or CLOSE refer to this specific cursor statement ).
--
Sami
On Fri, May 02, 2025 at 04:21:21PM -0500, Sami Imseih wrote:
postgres_fdw, as an example, in which cursor name get reused
for different queries. Notice below "c1" and "c2" is reused for different
queries, so now what underlying sql is FETCH, i.e. FETCH 100 FROM c1 referring
to? v2-0001 does not help us with the FETCH problem
because as I mentioned we don't have access to the underlying sql
( and parsing is even too early to do a portal lookup to find the
underlying sql to
base the queryId on). What v2-0001 will do is at least group the DECLARE CURSOR
statements together for cursors referencing the same query and reduce the #
of entries.
This case relies on postgres_fdw's GetCursorNumber() that assigns a
unique number for a cursor, ensuring uniqueness per connection within
a transaction, and the counter is reset at the end of the
transactions. So good point for this case that this hurts. If that
holds for the most common cases where this is seen as bloating pgss,
that brings some solid ground, especially more for applications that
use many cursor numbers in long-ish transactions states done under
postgres_fdw.
I'm still slightly worried about workloads where cursor names could be
used to track some balancing of this kind of activity, for example if
cursor names are fixed on a transaction-basis depending on the
application involved, as that would mean less visibility in the
information. Perhaps we would have more confidence by looking closer
at drivers or some ORMs that make use of cursors, seeing how the end
user would be impacted? That seems hard to measure, though..
The FETCH and CLOSE are already not clear to what underlying SQL
they are referring to, and there is not much chance to actually
improve that unless
we track a cursor queryId in pg_stat_statements ( at that point we can show that
FETCH or CLOSE refer to this specific cursor statement ).
I don't really have an issue for FETCH with the number as the name is
still around, but I'm equally worrying about the loss of information
for CLOSE that this new normalization would imply. Perhaps my worries
don't have a reason to exist here and I'm just a naturally-pessimistic
being.
--
Michael
postgres_fdw, as an example, in which cursor name get reused
for different queries. Notice below "c1" and "c2" is reused for different
queries, so now what underlying sql is FETCH, i.e. FETCH 100 FROM c1 referring
to? v2-0001 does not help us with the FETCH problem
because as I mentioned we don't have access to the underlying sql
( and parsing is even too early to do a portal lookup to find the
underlying sql to
base the queryId on). What v2-0001 will do is at least group the DECLARE CURSOR
statements together for cursors referencing the same query and reduce the #
of entries.This case relies on postgres_fdw's GetCursorNumber() that assigns a
unique number for a cursor, ensuring uniqueness per connection within
a transaction, and the counter is reset at the end of the
transactions. So good point for this case that this hurts. If that
holds for the most common cases where this is seen as bloating pgss,
that brings some solid ground, especially more for applications that
use many cursor numbers in long-ish transactions states done under
postgres_fdw.
Sorry for the delayed response. I’ve been thinking about this a bit, and
I agree that it’s really hard to get a good sense of the use cases out there.
postgres_fdw does have the issue of reusing cursor names, which renders
the stats essentially meaningless, in my opinion—not to mention it contributes
to excessive bloat. Looking at a driver like psycopg, explicit cursors
are supported
through the driver, but the user must define the name of the cursor,
so it's much
more controlled.
I really wonder if the right answer is to have a
pg_stat_statements.track_cursor_utility GUC that can toggle the
reporting of utility statements related to explicit cursors, while
still tracking the
underlying statements. I would even suggest making 'on' the default to
maintain the current behavior.
I don’t like that we have to introduce a new GUC for this,
but I can't think of a better alternative.
Thoughts?
--
Sami Imseih
Amazon Web Services (AWS)
The FETCH and CLOSE are already not clear to what underlying SQL
they are referring to, and there is not much chance to actually
improve that unless
we track a cursor queryId in pg_stat_statements ( at that point we can show that
FETCH or CLOSE refer to this specific cursor statement ).
I don't really have an issue for FETCH with the number as the name is
Since the FETCH case is clear-cut, here is a patch that normalizes variable
fetch sizes in a FETCH command. At a minimum, we can apply this patch.
I’ve also added tests in pg_stat_statements utility.sql to demonstrate
how queryIds
are grouped for the variants of the FETCH statement.
--
Sami Imseih
Amazon Web Services (AWS)
Attachments:
v2-0001-Normalize-variable-fetch-sizes-in-a-FETCH-command.patchapplication/octet-stream; name=v2-0001-Normalize-variable-fetch-sizes-in-a-FETCH-command.patchDownload
From fb1c79c481c03f99631a5c4560223db205226cd5 Mon Sep 17 00:00:00 2001
From: Sami Imseih <simseih@amazon.com>
Date: Mon, 2 Jun 2025 13:15:47 -0500
Subject: [PATCH v2 1/1] Normalize variable fetch sizes in a FETCH command
Prior to this patch, every FETCH call would generate a unique queryId.
This led to significant bloat in pg_stat_statements, as repeatedly calling
a specific cursor would result in a new queryId each time. For example,
FETCH 1 c1; and FETCH 2 c1; would produce different queryIds.
This patch improves the situation by normalizing the fetch size, so semantically
similar statements generate the same queryId. As a result, statements like the below,
which differ syntactically but have the same effect, will now share a single queryId:
FETCH FROM c1;
FETCH NEXT c1;
FETCH 1 c1;
Discussion: https://www.postgresql.org/message-id/flat/CAA5RZ0tA6LbHCg2qSS%2BKuM850BZC_%2BZgHV7Ug6BXw22TNyF%2BMA%40mail.gmail.com
---
.../pg_stat_statements/expected/cursors.out | 4 +-
.../expected/level_tracking.out | 4 +-
.../pg_stat_statements/expected/utility.out | 163 +++++++++++++++++-
contrib/pg_stat_statements/sql/utility.sql | 45 +++++
src/backend/parser/gram.y | 16 ++
src/include/nodes/parsenodes.h | 4 +-
6 files changed, 227 insertions(+), 9 deletions(-)
diff --git a/contrib/pg_stat_statements/expected/cursors.out b/contrib/pg_stat_statements/expected/cursors.out
index 0fc4b2c098d0..6781f99ea03e 100644
--- a/contrib/pg_stat_statements/expected/cursors.out
+++ b/contrib/pg_stat_statements/expected/cursors.out
@@ -57,8 +57,8 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 0 | COMMIT
1 | 0 | DECLARE cursor_stats_1 CURSOR WITH HOLD FOR SELECT $1
1 | 0 | DECLARE cursor_stats_2 CURSOR WITH HOLD FOR SELECT $1
- 1 | 1 | FETCH 1 IN cursor_stats_1
- 1 | 1 | FETCH 1 IN cursor_stats_2
+ 1 | 1 | FETCH $1 IN cursor_stats_1
+ 1 | 1 | FETCH $1 IN cursor_stats_2
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(9 rows)
diff --git a/contrib/pg_stat_statements/expected/level_tracking.out b/contrib/pg_stat_statements/expected/level_tracking.out
index 75e785e1719e..ce4bd3a823d8 100644
--- a/contrib/pg_stat_statements/expected/level_tracking.out
+++ b/contrib/pg_stat_statements/expected/level_tracking.out
@@ -1142,7 +1142,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | CLOSE foocur
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * from stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | FETCH FORWARD $1 FROM foocur
f | 1 | SELECT * from stats_track_tab
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(7 rows)
@@ -1172,7 +1172,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | CLOSE foocur
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * FROM stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | FETCH FORWARD $1 FROM foocur
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(6 rows)
diff --git a/contrib/pg_stat_statements/expected/utility.out b/contrib/pg_stat_statements/expected/utility.out
index aa4f0f7e6280..ae6fad96ebbe 100644
--- a/contrib/pg_stat_statements/expected/utility.out
+++ b/contrib/pg_stat_statements/expected/utility.out
@@ -702,13 +702,11 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 13 | CREATE MATERIALIZED VIEW pgss_matv AS SELECT * FROM pgss_ctas
1 | 10 | CREATE TABLE pgss_ctas AS SELECT a, $1 b FROM generate_series($2, $3) a
1 | 0 | DECLARE pgss_cursor CURSOR FOR SELECT * FROM pgss_matv
- 1 | 5 | FETCH FORWARD 5 pgss_cursor
- 1 | 7 | FETCH FORWARD ALL pgss_cursor
- 1 | 1 | FETCH NEXT pgss_cursor
+ 3 | 13 | FETCH NEXT pgss_cursor
1 | 13 | REFRESH MATERIALIZED VIEW pgss_matv
1 | 10 | SELECT generate_series($1, $2) c INTO pgss_select_into
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
-(12 rows)
+(10 rows)
DROP MATERIALIZED VIEW pgss_matv;
DROP TABLE pgss_ctas;
@@ -719,7 +717,164 @@ SELECT pg_stat_statements_reset() IS NOT NULL AS t;
t
(1 row)
+-- Normalization of FETCH statements
+-- this group will have the same queryId
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+FETCH pgss_cursor;
+--
+(1 row)
+
+FETCH FROM pgss_cursor;
+--
+(1 row)
+
+FETCH NEXT pgss_cursor;
+--
+(1 row)
+
+FETCH 1 pgss_cursor;
+--
+(1 row)
+
+FETCH 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH 3 pgss_cursor;
+--
+(3 rows)
+
+FETCH ALL pgss_cursor;
+--
+(41 rows)
+
+FETCH FORWARD pgss_cursor;
+--
+(0 rows)
+
+FETCH FORWARD 1 pgss_cursor;
+--
+(0 rows)
+
+FETCH FORWARD 2 pgss_cursor;
+--
+(0 rows)
+
+FETCH FORWARD 3 pgss_cursor;
+--
+(0 rows)
+
+FETCH FORWARD ALL pgss_cursor;
+--
+(0 rows)
+
+-- this next group will have the same queryId
+FETCH PRIOR pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD 1 pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH BACKWARD 3 pgss_cursor;
+--
+(3 rows)
+
+FETCH BACKWARD ALL pgss_cursor;
+--
+(43 rows)
+
+-- this next group will have the same queryId
+FETCH FIRST pgss_cursor;
+--
+(1 row)
+
+FETCH LAST pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 1 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 2 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 3 pgss_cursor;
+--
+(1 row)
+
+-- this next group will have the same queryId
+FETCH RELATIVE 1 pgss_cursor;
+--
+(1 row)
+
+FETCH RELATIVE 2 pgss_cursor;
+--
+(1 row)
+
+FETCH RELATIVE 3 pgss_cursor;
+--
+(1 row)
+
+COMMIT;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
+ calls | query
+-------+--------------------------------------------------------------------
+ 1 | BEGIN
+ 1 | COMMIT
+ 1 | DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series($1, $2)
+ 5 | FETCH FIRST pgss_cursor
+ 5 | FETCH PRIOR pgss_cursor
+ 3 | FETCH RELATIVE $1 pgss_cursor
+ 12 | FETCH pgss_cursor
+ 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
+(8 rows)
+
+-- check that these statements normalize the fetch size
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+FETCH 1 pgss_cursor;
+--
+(1 row)
+
+FETCH 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH 3 pgss_cursor;
+--
+(3 rows)
+
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
+ calls | query
+-------+--------------------------------------------------------------------
+ 1 | BEGIN
+ 1 | DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series($1, $2)
+ 3 | FETCH $1 pgss_cursor
+ 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
+(4 rows)
+
-- Special cases. Keep these ones at the end to avoid conflicts.
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
SET SCHEMA 'foo';
SET SCHEMA 'public';
RESET ALL;
diff --git a/contrib/pg_stat_statements/sql/utility.sql b/contrib/pg_stat_statements/sql/utility.sql
index dd97203c2102..db14f389eee8 100644
--- a/contrib/pg_stat_statements/sql/utility.sql
+++ b/contrib/pg_stat_statements/sql/utility.sql
@@ -365,7 +365,52 @@ DROP TABLE pgss_select_into;
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+-- Normalization of FETCH statements
+-- this group will have the same queryId
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+FETCH pgss_cursor;
+FETCH FROM pgss_cursor;
+FETCH NEXT pgss_cursor;
+FETCH 1 pgss_cursor;
+FETCH 2 pgss_cursor;
+FETCH 3 pgss_cursor;
+FETCH ALL pgss_cursor;
+FETCH FORWARD pgss_cursor;
+FETCH FORWARD 1 pgss_cursor;
+FETCH FORWARD 2 pgss_cursor;
+FETCH FORWARD 3 pgss_cursor;
+FETCH FORWARD ALL pgss_cursor;
+-- this next group will have the same queryId
+FETCH PRIOR pgss_cursor;
+FETCH BACKWARD 1 pgss_cursor;
+FETCH BACKWARD 2 pgss_cursor;
+FETCH BACKWARD 3 pgss_cursor;
+FETCH BACKWARD ALL pgss_cursor;
+-- this next group will have the same queryId
+FETCH FIRST pgss_cursor;
+FETCH LAST pgss_cursor;
+FETCH ABSOLUTE 1 pgss_cursor;
+FETCH ABSOLUTE 2 pgss_cursor;
+FETCH ABSOLUTE 3 pgss_cursor;
+-- this next group will have the same queryId
+FETCH RELATIVE 1 pgss_cursor;
+FETCH RELATIVE 2 pgss_cursor;
+FETCH RELATIVE 3 pgss_cursor;
+COMMIT;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
+-- check that these statements normalize the fetch size
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+FETCH 1 pgss_cursor;
+FETCH 2 pgss_cursor;
+FETCH 3 pgss_cursor;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
+
+
-- Special cases. Keep these ones at the end to avoid conflicts.
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
SET SCHEMA 'foo';
SET SCHEMA 'public';
RESET ALL;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0b5652071d11..843dcc87a48b 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -7479,6 +7479,7 @@ fetch_args: cursor_name
n->portalname = $1;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| from_in cursor_name
@@ -7488,6 +7489,7 @@ fetch_args: cursor_name
n->portalname = $2;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| NEXT opt_from_in cursor_name
@@ -7497,6 +7499,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| PRIOR opt_from_in cursor_name
@@ -7506,6 +7509,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| FIRST_P opt_from_in cursor_name
@@ -7515,6 +7519,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| LAST_P opt_from_in cursor_name
@@ -7524,6 +7529,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = -1;
+ n->location = -1;
$$ = (Node *) n;
}
| ABSOLUTE_P SignedIconst opt_from_in cursor_name
@@ -7533,6 +7539,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_ABSOLUTE;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| RELATIVE_P SignedIconst opt_from_in cursor_name
@@ -7542,6 +7549,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_RELATIVE;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| SignedIconst opt_from_in cursor_name
@@ -7551,6 +7559,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = $1;
+ n->location = @1;
$$ = (Node *) n;
}
| ALL opt_from_in cursor_name
@@ -7560,6 +7569,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
$$ = (Node *) n;
}
| FORWARD opt_from_in cursor_name
@@ -7569,6 +7579,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| FORWARD SignedIconst opt_from_in cursor_name
@@ -7578,6 +7589,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| FORWARD ALL opt_from_in cursor_name
@@ -7587,6 +7599,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
$$ = (Node *) n;
}
| BACKWARD opt_from_in cursor_name
@@ -7596,6 +7609,7 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
$$ = (Node *) n;
}
| BACKWARD SignedIconst opt_from_in cursor_name
@@ -7605,6 +7619,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = $2;
+ n->location = @2;
$$ = (Node *) n;
}
| BACKWARD ALL opt_from_in cursor_name
@@ -7614,6 +7629,7 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
$$ = (Node *) n;
}
;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index dd00ab420b8a..02bfce1ff38e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3428,9 +3428,11 @@ typedef struct FetchStmt
{
NodeTag type;
FetchDirection direction; /* see above */
- long howMany; /* number of rows, or position argument */
+ /* number of rows, or position argument */
+ long howMany pg_node_attr(query_jumble_ignore);
char *portalname; /* name of portal (cursor) */
bool ismove; /* true if MOVE */
+ ParseLoc location pg_node_attr(query_jumble_location);
} FetchStmt;
/* ----------------------
--
2.39.5 (Apple Git-154)
On Mon, Jun 02, 2025 at 02:44:36PM -0500, Sami Imseih wrote:
Since the FETCH case is clear-cut, here is a patch that normalizes variable
fetch sizes in a FETCH command. At a minimum, we can apply this patch.
I’ve also added tests in pg_stat_statements utility.sql to demonstrate
how queryIds
are grouped for the variants of the FETCH statement.
{
NodeTag type;
FetchDirection direction; /* see above */
- long howMany; /* number of rows, or position argument */
+ /* number of rows, or position argument */
+ long howMany pg_node_attr(query_jumble_ignore);
char *portalname; /* name of portal (cursor) */
bool ismove; /* true if MOVE */
+ ParseLoc location pg_node_attr(query_jumble_location);
} FetchStmt;
In terms of silencing the numbers, that's fine by me. Now one issue
is that this also masks FETCH_ALL which is a specific keyword in the
grammar.
Should we offer something more consistent with DeallocateStmt, where
we have a boolean flag that would be set when ALL is specified,
included in the jumbling? This would mean two separate entries: one
for the constants and one for ALL.
--
Michael
Should we offer something more consistent with DeallocateStmt, where
we have a boolean flag that would be set when ALL is specified,
included in the jumbling? This would mean two separate entries: one
for the constants and one for ALL.
Hmm, we could do that to differentiate the keyword ALL. I had a thought
earlier about differentiating the other keywords as well: FIRST, LAST,
BACKWARD, FORWARD, and ABSOLUTE. Initially, I thought it might
be a bit too much, but I do see the merit in this approach, as these are
syntactically different from their numeric counterparts.
We can accomplish this with just an extra field in FetchStmt, where each
of these keywords gets a distinct value.
What do you think?
--
Sami
Hmm, we could do that to differentiate the keyword ALL. I had a thought
earlier about differentiating the other keywords as well: FIRST, LAST,
BACKWARD, FORWARD, and ABSOLUTE. Initially, I thought it might
be a bit too much, but I do see the merit in this approach, as these are
syntactically different from their numeric counterparts.We can accomplish this with just an extra field in FetchStmt, where each
of these keywords gets a distinct value.What do you think?
v3 is what I'm thinking: In FetchStmt, introduce a new field called
explicit_direction (maybe there's a better name?), which is an int that
we can assign a value to in gram.c. The value will be 0 if no explicit
direction is used. Otherwise, each of the explicit directions (i.e., FIRST,
LAST, BACKWARD) will be assigned a unique value. FORWARD ALL and
BACKWARD ALL will be treated as distinct from FORWARD and BACKWARD.
I considered introducing an enum for these explicit direction values, but
didn’t find it particularly useful. If you think it would be beneficial,
I can define an enum in parsenodes.h
I also slightly reorganized the fetch_args so the like values for
n->explicit_direction are nearby each other. It does not change
the behavior.
Also, modified the tests to match and some additional code comments.
--
Sami Imseih
Amazon Web Services (AWS)
Attachments:
v3-0001-Normalize-variable-fetch-sizes-in-a-FETCH-command.patchapplication/octet-stream; name=v3-0001-Normalize-variable-fetch-sizes-in-a-FETCH-command.patchDownload
From a36ba8b99e9099ca45c065a01af80a7029304acd Mon Sep 17 00:00:00 2001
From: Ubuntu <ubuntu@ip-172-31-38-230.ec2.internal>
Date: Tue, 3 Jun 2025 17:58:19 +0000
Subject: [PATCH v3 1/1] Normalize variable fetch sizes in a FETCH command
Prior to this patch, every FETCH call would generate a unique queryId.
This led to significant bloat in pg_stat_statements, as repeatedly calling
a specific cursor would result in a new queryId each time. For example,
FETCH 1 c1; and FETCH 2 c1; would produce different queryIds.
This patch improves the situation by normalizing the fetch size, so semantically
similar statements generate the same queryId. As a result, statements like the below,
which differ syntactically but have the same effect, will now share a single queryId:
FETCH FROM c1;
FETCH NEXT c1;
FETCH 1 c1;
Discussion: https://www.postgresql.org/message-id/flat/CAA5RZ0tA6LbHCg2qSS%2BKuM850BZC_%2BZgHV7Ug6BXw22TNyF%2BMA%40mail.gmail.com
---
.../pg_stat_statements/expected/cursors.out | 4 +-
.../expected/level_tracking.out | 4 +-
.../pg_stat_statements/expected/utility.out | 165 +++++++++++++++++-
contrib/pg_stat_statements/sql/utility.sql | 49 ++++++
src/backend/parser/gram.y | 58 +++++-
src/include/nodes/parsenodes.h | 12 +-
6 files changed, 277 insertions(+), 15 deletions(-)
diff --git a/contrib/pg_stat_statements/expected/cursors.out b/contrib/pg_stat_statements/expected/cursors.out
index 0fc4b2c098d..6781f99ea03 100644
--- a/contrib/pg_stat_statements/expected/cursors.out
+++ b/contrib/pg_stat_statements/expected/cursors.out
@@ -57,8 +57,8 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 0 | COMMIT
1 | 0 | DECLARE cursor_stats_1 CURSOR WITH HOLD FOR SELECT $1
1 | 0 | DECLARE cursor_stats_2 CURSOR WITH HOLD FOR SELECT $1
- 1 | 1 | FETCH 1 IN cursor_stats_1
- 1 | 1 | FETCH 1 IN cursor_stats_2
+ 1 | 1 | FETCH $1 IN cursor_stats_1
+ 1 | 1 | FETCH $1 IN cursor_stats_2
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(9 rows)
diff --git a/contrib/pg_stat_statements/expected/level_tracking.out b/contrib/pg_stat_statements/expected/level_tracking.out
index 75e785e1719..ce4bd3a823d 100644
--- a/contrib/pg_stat_statements/expected/level_tracking.out
+++ b/contrib/pg_stat_statements/expected/level_tracking.out
@@ -1142,7 +1142,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | CLOSE foocur
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * from stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | FETCH FORWARD $1 FROM foocur
f | 1 | SELECT * from stats_track_tab
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(7 rows)
@@ -1172,7 +1172,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | CLOSE foocur
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * FROM stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | FETCH FORWARD $1 FROM foocur
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(6 rows)
diff --git a/contrib/pg_stat_statements/expected/utility.out b/contrib/pg_stat_statements/expected/utility.out
index aa4f0f7e628..5b623e0a095 100644
--- a/contrib/pg_stat_statements/expected/utility.out
+++ b/contrib/pg_stat_statements/expected/utility.out
@@ -702,7 +702,7 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 13 | CREATE MATERIALIZED VIEW pgss_matv AS SELECT * FROM pgss_ctas
1 | 10 | CREATE TABLE pgss_ctas AS SELECT a, $1 b FROM generate_series($2, $3) a
1 | 0 | DECLARE pgss_cursor CURSOR FOR SELECT * FROM pgss_matv
- 1 | 5 | FETCH FORWARD 5 pgss_cursor
+ 1 | 5 | FETCH FORWARD $1 pgss_cursor
1 | 7 | FETCH FORWARD ALL pgss_cursor
1 | 1 | FETCH NEXT pgss_cursor
1 | 13 | REFRESH MATERIALIZED VIEW pgss_matv
@@ -719,7 +719,170 @@ SELECT pg_stat_statements_reset() IS NOT NULL AS t;
t
(1 row)
+-- Normalization of FETCH statements
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+-- implicit directions
+FETCH pgss_cursor;
+--
+(1 row)
+
+FETCH 1 pgss_cursor;
+--
+(1 row)
+
+FETCH 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH 3 pgss_cursor;
+--
+(3 rows)
+
+FETCH -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit NEXT
+FETCH NEXT pgss_cursor;
+--
+(1 row)
+
+-- explicit PRIOR
+FETCH PRIOR pgss_cursor;
+--
+(1 row)
+
+-- explicit FIRST
+FETCH FIRST pgss_cursor;
+--
+(1 row)
+
+-- explicit LAST
+FETCH LAST pgss_cursor;
+--
+(1 row)
+
+-- explicit ABSOLUTE
+FETCH ABSOLUTE 1 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 2 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 3 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit RELATIVE
+FETCH RELATIVE 1 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE 2 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE 3 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit FORWARD
+FETCH ALL pgss_cursor;
+--
+(0 rows)
+
+-- explicit FORWARD ALL
+FETCH FORWARD ALL pgss_cursor;
+--
+(0 rows)
+
+-- explicit BACKWARD ALL
+FETCH BACKWARD ALL pgss_cursor;
+--
+(50 rows)
+
+-- explicit FETCH FORWARD
+FETCH FORWARD pgss_cursor;
+--
+(1 row)
+
+FETCH FORWARD 1 pgss_cursor;
+--
+(1 row)
+
+FETCH FORWARD 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH FORWARD 3 pgss_cursor;
+--
+(3 rows)
+
+FETCH FORWARD -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit FETCH BACKWARD
+FETCH BACKWARD pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD 1 pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH BACKWARD 3 pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD -1 pgss_cursor;
+--
+(1 row)
+
+COMMIT;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
+ calls | query
+-------+--------------------------------------------------------------------
+ 1 | BEGIN
+ 1 | COMMIT
+ 1 | DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series($1, $2)
+ 4 | FETCH ABSOLUTE $1 pgss_cursor
+ 1 | FETCH ALL pgss_cursor
+ 1 | FETCH BACKWARD ALL pgss_cursor
+ 5 | FETCH BACKWARD pgss_cursor
+ 1 | FETCH FIRST pgss_cursor
+ 1 | FETCH FORWARD ALL pgss_cursor
+ 5 | FETCH FORWARD pgss_cursor
+ 1 | FETCH LAST pgss_cursor
+ 1 | FETCH NEXT pgss_cursor
+ 1 | FETCH PRIOR pgss_cursor
+ 4 | FETCH RELATIVE $1 pgss_cursor
+ 5 | FETCH pgss_cursor
+ 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
+(16 rows)
+
-- Special cases. Keep these ones at the end to avoid conflicts.
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
SET SCHEMA 'foo';
SET SCHEMA 'public';
RESET ALL;
diff --git a/contrib/pg_stat_statements/sql/utility.sql b/contrib/pg_stat_statements/sql/utility.sql
index dd97203c210..8fcd7d2983f 100644
--- a/contrib/pg_stat_statements/sql/utility.sql
+++ b/contrib/pg_stat_statements/sql/utility.sql
@@ -365,7 +365,56 @@ DROP TABLE pgss_select_into;
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+-- Normalization of FETCH statements
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+-- implicit directions
+FETCH pgss_cursor;
+FETCH 1 pgss_cursor;
+FETCH 2 pgss_cursor;
+FETCH 3 pgss_cursor;
+FETCH -1 pgss_cursor;
+-- explicit NEXT
+FETCH NEXT pgss_cursor;
+-- explicit PRIOR
+FETCH PRIOR pgss_cursor;
+-- explicit FIRST
+FETCH FIRST pgss_cursor;
+-- explicit LAST
+FETCH LAST pgss_cursor;
+-- explicit ABSOLUTE
+FETCH ABSOLUTE 1 pgss_cursor;
+FETCH ABSOLUTE 2 pgss_cursor;
+FETCH ABSOLUTE 3 pgss_cursor;
+FETCH ABSOLUTE -1 pgss_cursor;
+-- explicit RELATIVE
+FETCH RELATIVE 1 pgss_cursor;
+FETCH RELATIVE 2 pgss_cursor;
+FETCH RELATIVE 3 pgss_cursor;
+FETCH RELATIVE -1 pgss_cursor;
+-- explicit FORWARD
+FETCH ALL pgss_cursor;
+-- explicit FORWARD ALL
+FETCH FORWARD ALL pgss_cursor;
+-- explicit BACKWARD ALL
+FETCH BACKWARD ALL pgss_cursor;
+-- explicit FETCH FORWARD
+FETCH FORWARD pgss_cursor;
+FETCH FORWARD 1 pgss_cursor;
+FETCH FORWARD 2 pgss_cursor;
+FETCH FORWARD 3 pgss_cursor;
+FETCH FORWARD -1 pgss_cursor;
+-- explicit FETCH BACKWARD
+FETCH BACKWARD pgss_cursor;
+FETCH BACKWARD 1 pgss_cursor;
+FETCH BACKWARD 2 pgss_cursor;
+FETCH BACKWARD 3 pgss_cursor;
+FETCH BACKWARD -1 pgss_cursor;
+COMMIT;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
+
-- Special cases. Keep these ones at the end to avoid conflicts.
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
SET SCHEMA 'foo';
SET SCHEMA 'public';
RESET ALL;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0b5652071d1..448897e7971 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -7472,6 +7472,14 @@ FetchStmt: FETCH fetch_args
}
;
+/*
+ * Query jumbling should distinguish between explicit directions
+ * (e.g., FETCH FORWARD) and implicit directions (e.g., FETCH 10).
+ * For each specified direction keyword, assign a unique value greater
+ * than 0 to n->explicit_direction. If no keyword is specified,
+ * assign 0 to n->explicit_direction. Treat keywords supplied with or
+ * without a fetch size the same.
+ */
fetch_args: cursor_name
{
FetchStmt *n = makeNode(FetchStmt);
@@ -7479,6 +7487,8 @@ fetch_args: cursor_name
n->portalname = $1;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->explicit_direction = 0;
$$ = (Node *) n;
}
| from_in cursor_name
@@ -7488,6 +7498,19 @@ fetch_args: cursor_name
n->portalname = $2;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->explicit_direction = 0;
+ $$ = (Node *) n;
+ }
+ | SignedIconst opt_from_in cursor_name
+ {
+ FetchStmt *n = makeNode(FetchStmt);
+
+ n->portalname = $3;
+ n->direction = FETCH_FORWARD;
+ n->howMany = $1;
+ n->location = @1;
+ n->explicit_direction = 0;
$$ = (Node *) n;
}
| NEXT opt_from_in cursor_name
@@ -7497,6 +7520,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->explicit_direction = 1;
$$ = (Node *) n;
}
| PRIOR opt_from_in cursor_name
@@ -7506,6 +7531,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
+ n->explicit_direction = 2;
$$ = (Node *) n;
}
| FIRST_P opt_from_in cursor_name
@@ -7515,6 +7542,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = 1;
+ n->location = -1;
+ n->explicit_direction = 3;
$$ = (Node *) n;
}
| LAST_P opt_from_in cursor_name
@@ -7524,6 +7553,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = -1;
+ n->location = -1;
+ n->explicit_direction = 4;
$$ = (Node *) n;
}
| ABSOLUTE_P SignedIconst opt_from_in cursor_name
@@ -7533,6 +7564,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_ABSOLUTE;
n->howMany = $2;
+ n->location = @2;
+ n->explicit_direction = 5;
$$ = (Node *) n;
}
| RELATIVE_P SignedIconst opt_from_in cursor_name
@@ -7542,15 +7575,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_RELATIVE;
n->howMany = $2;
- $$ = (Node *) n;
- }
- | SignedIconst opt_from_in cursor_name
- {
- FetchStmt *n = makeNode(FetchStmt);
-
- n->portalname = $3;
- n->direction = FETCH_FORWARD;
- n->howMany = $1;
+ n->location = @2;
+ n->explicit_direction = 6;
$$ = (Node *) n;
}
| ALL opt_from_in cursor_name
@@ -7560,6 +7586,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->explicit_direction = 7;
$$ = (Node *) n;
}
| FORWARD opt_from_in cursor_name
@@ -7569,6 +7597,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->explicit_direction = 8;
$$ = (Node *) n;
}
| FORWARD SignedIconst opt_from_in cursor_name
@@ -7578,6 +7608,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = $2;
+ n->location = @2;
+ n->explicit_direction = 8;
$$ = (Node *) n;
}
| FORWARD ALL opt_from_in cursor_name
@@ -7587,6 +7619,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->explicit_direction = 9;
$$ = (Node *) n;
}
| BACKWARD opt_from_in cursor_name
@@ -7596,6 +7630,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
+ n->explicit_direction = 10;
$$ = (Node *) n;
}
| BACKWARD SignedIconst opt_from_in cursor_name
@@ -7605,6 +7641,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = $2;
+ n->location = @2;
+ n->explicit_direction = 10;
$$ = (Node *) n;
}
| BACKWARD ALL opt_from_in cursor_name
@@ -7614,6 +7652,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->explicit_direction = 11;
$$ = (Node *) n;
}
;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index dd00ab420b8..293d7711ef9 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3428,9 +3428,19 @@ typedef struct FetchStmt
{
NodeTag type;
FetchDirection direction; /* see above */
- long howMany; /* number of rows, or position argument */
+ /* number of rows, or position argument */
+ long howMany pg_node_attr(query_jumble_ignore);
char *portalname; /* name of portal (cursor) */
bool ismove; /* true if MOVE */
+
+ /*
+ * If an explicit direction (i.e., FETCH FORWARD) is used, set this field
+ * to distinguish it from its numeric counterpart (i.e., FETCH 10). This
+ * value is set only within gram.y.
+ */
+ int explicit_direction;
+
+ ParseLoc location pg_node_attr(query_jumble_location);
} FetchStmt;
/* ----------------------
--
2.43.0
On Tue, Jun 03, 2025 at 01:04:36PM -0500, Sami Imseih wrote:
v3 is what I'm thinking: In FetchStmt, introduce a new field called
explicit_direction (maybe there's a better name?), which is an int that
we can assign a value to in gram.c. The value will be 0 if no explicit
direction is used. Otherwise, each of the explicit directions (i.e., FIRST,
LAST, BACKWARD) will be assigned a unique value. FORWARD ALL and
BACKWARD ALL will be treated as distinct from FORWARD and BACKWARD.
Hmm. I was not sure if we'd really need to get down to that, as most
of the grammar keywords have the same parsed meaning, but there's a
good point with LAST for example that uses a negative value for
howMany. If we silence the number, LAST would map with everything
else that has FETCH_ABSOLUTE. That would be confusing.
I considered introducing an enum for these explicit direction values, but
didn’t find it particularly useful. If you think it would be beneficial,
I can define an enum in parsenodes.h
An enum would shine here IMO, because the value could be
self-documented and one would not need to guess what each integer
value means. That could be something like a direction_keyword, filled
with FETCH_KEYWORD_LAST, FETCH_KEYWORD_BACKWARD, etc.
--
Michael
Hmm. I was not sure if we'd really need to get down to that, as most
of the grammar keywords have the same parsed meaning, but there's a
good point with LAST for example that uses a negative value for
howMany. If we silence the number, LAST would map with everything
else that has FETCH_ABSOLUTE. That would be confusing.
yes, that is another good case. I think distinguishing between the
various keywords makes the most sense.
I considered introducing an enum for these explicit direction values, but
didn’t find it particularly useful. If you think it would be beneficial,
I can define an enum in parsenodes.hAn enum would shine here IMO, because the value could be
self-documented and one would not need to guess what each integer
value means. That could be something like a direction_keyword, filled
with FETCH_KEYWORD_LAST, FETCH_KEYWORD_BACKWARD, etc.
Done in v4.
--
Sami
Attachments:
v4-0001-Normalize-variable-fetch-sizes-in-a-FETCH-command.patchapplication/octet-stream; name=v4-0001-Normalize-variable-fetch-sizes-in-a-FETCH-command.patchDownload
From c54b308feef684ea169002944241e5b8d7e2bd69 Mon Sep 17 00:00:00 2001
From: Ubuntu <ubuntu@ip-172-31-38-230.ec2.internal>
Date: Tue, 3 Jun 2025 17:58:19 +0000
Subject: [PATCH v4 1/1] Normalize variable fetch sizes in a FETCH command
Prior to this patch, every FETCH call would generate a unique queryId.
This led to significant bloat in pg_stat_statements, as repeatedly calling
a specific cursor would result in a new queryId each time. For example,
FETCH 1 c1; and FETCH 2 c1; would produce different queryIds.
This patch improves the situation by normalizing the fetch size, so semantically
similar statements generate the same queryId. As a result, statements like the below,
which differ syntactically but have the same effect, will now share a single queryId:
FETCH FROM c1;
FETCH NEXT c1;
FETCH 1 c1;
Discussion: https://www.postgresql.org/message-id/flat/CAA5RZ0tA6LbHCg2qSS%2BKuM850BZC_%2BZgHV7Ug6BXw22TNyF%2BMA%40mail.gmail.com
---
.../pg_stat_statements/expected/cursors.out | 4 +-
.../expected/level_tracking.out | 4 +-
.../pg_stat_statements/expected/utility.out | 165 +++++++++++++++++-
contrib/pg_stat_statements/sql/utility.sql | 49 ++++++
src/backend/parser/gram.y | 50 +++++-
src/include/nodes/parsenodes.h | 28 ++-
6 files changed, 285 insertions(+), 15 deletions(-)
diff --git a/contrib/pg_stat_statements/expected/cursors.out b/contrib/pg_stat_statements/expected/cursors.out
index 0fc4b2c098d..6781f99ea03 100644
--- a/contrib/pg_stat_statements/expected/cursors.out
+++ b/contrib/pg_stat_statements/expected/cursors.out
@@ -57,8 +57,8 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 0 | COMMIT
1 | 0 | DECLARE cursor_stats_1 CURSOR WITH HOLD FOR SELECT $1
1 | 0 | DECLARE cursor_stats_2 CURSOR WITH HOLD FOR SELECT $1
- 1 | 1 | FETCH 1 IN cursor_stats_1
- 1 | 1 | FETCH 1 IN cursor_stats_2
+ 1 | 1 | FETCH $1 IN cursor_stats_1
+ 1 | 1 | FETCH $1 IN cursor_stats_2
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(9 rows)
diff --git a/contrib/pg_stat_statements/expected/level_tracking.out b/contrib/pg_stat_statements/expected/level_tracking.out
index 75e785e1719..ce4bd3a823d 100644
--- a/contrib/pg_stat_statements/expected/level_tracking.out
+++ b/contrib/pg_stat_statements/expected/level_tracking.out
@@ -1142,7 +1142,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | CLOSE foocur
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * from stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | FETCH FORWARD $1 FROM foocur
f | 1 | SELECT * from stats_track_tab
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(7 rows)
@@ -1172,7 +1172,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | CLOSE foocur
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * FROM stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | FETCH FORWARD $1 FROM foocur
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(6 rows)
diff --git a/contrib/pg_stat_statements/expected/utility.out b/contrib/pg_stat_statements/expected/utility.out
index aa4f0f7e628..5b623e0a095 100644
--- a/contrib/pg_stat_statements/expected/utility.out
+++ b/contrib/pg_stat_statements/expected/utility.out
@@ -702,7 +702,7 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 13 | CREATE MATERIALIZED VIEW pgss_matv AS SELECT * FROM pgss_ctas
1 | 10 | CREATE TABLE pgss_ctas AS SELECT a, $1 b FROM generate_series($2, $3) a
1 | 0 | DECLARE pgss_cursor CURSOR FOR SELECT * FROM pgss_matv
- 1 | 5 | FETCH FORWARD 5 pgss_cursor
+ 1 | 5 | FETCH FORWARD $1 pgss_cursor
1 | 7 | FETCH FORWARD ALL pgss_cursor
1 | 1 | FETCH NEXT pgss_cursor
1 | 13 | REFRESH MATERIALIZED VIEW pgss_matv
@@ -719,7 +719,170 @@ SELECT pg_stat_statements_reset() IS NOT NULL AS t;
t
(1 row)
+-- Normalization of FETCH statements
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+-- implicit directions
+FETCH pgss_cursor;
+--
+(1 row)
+
+FETCH 1 pgss_cursor;
+--
+(1 row)
+
+FETCH 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH 3 pgss_cursor;
+--
+(3 rows)
+
+FETCH -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit NEXT
+FETCH NEXT pgss_cursor;
+--
+(1 row)
+
+-- explicit PRIOR
+FETCH PRIOR pgss_cursor;
+--
+(1 row)
+
+-- explicit FIRST
+FETCH FIRST pgss_cursor;
+--
+(1 row)
+
+-- explicit LAST
+FETCH LAST pgss_cursor;
+--
+(1 row)
+
+-- explicit ABSOLUTE
+FETCH ABSOLUTE 1 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 2 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 3 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit RELATIVE
+FETCH RELATIVE 1 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE 2 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE 3 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit FORWARD
+FETCH ALL pgss_cursor;
+--
+(0 rows)
+
+-- explicit FORWARD ALL
+FETCH FORWARD ALL pgss_cursor;
+--
+(0 rows)
+
+-- explicit BACKWARD ALL
+FETCH BACKWARD ALL pgss_cursor;
+--
+(50 rows)
+
+-- explicit FETCH FORWARD
+FETCH FORWARD pgss_cursor;
+--
+(1 row)
+
+FETCH FORWARD 1 pgss_cursor;
+--
+(1 row)
+
+FETCH FORWARD 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH FORWARD 3 pgss_cursor;
+--
+(3 rows)
+
+FETCH FORWARD -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit FETCH BACKWARD
+FETCH BACKWARD pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD 1 pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH BACKWARD 3 pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD -1 pgss_cursor;
+--
+(1 row)
+
+COMMIT;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
+ calls | query
+-------+--------------------------------------------------------------------
+ 1 | BEGIN
+ 1 | COMMIT
+ 1 | DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series($1, $2)
+ 4 | FETCH ABSOLUTE $1 pgss_cursor
+ 1 | FETCH ALL pgss_cursor
+ 1 | FETCH BACKWARD ALL pgss_cursor
+ 5 | FETCH BACKWARD pgss_cursor
+ 1 | FETCH FIRST pgss_cursor
+ 1 | FETCH FORWARD ALL pgss_cursor
+ 5 | FETCH FORWARD pgss_cursor
+ 1 | FETCH LAST pgss_cursor
+ 1 | FETCH NEXT pgss_cursor
+ 1 | FETCH PRIOR pgss_cursor
+ 4 | FETCH RELATIVE $1 pgss_cursor
+ 5 | FETCH pgss_cursor
+ 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
+(16 rows)
+
-- Special cases. Keep these ones at the end to avoid conflicts.
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
SET SCHEMA 'foo';
SET SCHEMA 'public';
RESET ALL;
diff --git a/contrib/pg_stat_statements/sql/utility.sql b/contrib/pg_stat_statements/sql/utility.sql
index dd97203c210..8fcd7d2983f 100644
--- a/contrib/pg_stat_statements/sql/utility.sql
+++ b/contrib/pg_stat_statements/sql/utility.sql
@@ -365,7 +365,56 @@ DROP TABLE pgss_select_into;
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+-- Normalization of FETCH statements
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+-- implicit directions
+FETCH pgss_cursor;
+FETCH 1 pgss_cursor;
+FETCH 2 pgss_cursor;
+FETCH 3 pgss_cursor;
+FETCH -1 pgss_cursor;
+-- explicit NEXT
+FETCH NEXT pgss_cursor;
+-- explicit PRIOR
+FETCH PRIOR pgss_cursor;
+-- explicit FIRST
+FETCH FIRST pgss_cursor;
+-- explicit LAST
+FETCH LAST pgss_cursor;
+-- explicit ABSOLUTE
+FETCH ABSOLUTE 1 pgss_cursor;
+FETCH ABSOLUTE 2 pgss_cursor;
+FETCH ABSOLUTE 3 pgss_cursor;
+FETCH ABSOLUTE -1 pgss_cursor;
+-- explicit RELATIVE
+FETCH RELATIVE 1 pgss_cursor;
+FETCH RELATIVE 2 pgss_cursor;
+FETCH RELATIVE 3 pgss_cursor;
+FETCH RELATIVE -1 pgss_cursor;
+-- explicit FORWARD
+FETCH ALL pgss_cursor;
+-- explicit FORWARD ALL
+FETCH FORWARD ALL pgss_cursor;
+-- explicit BACKWARD ALL
+FETCH BACKWARD ALL pgss_cursor;
+-- explicit FETCH FORWARD
+FETCH FORWARD pgss_cursor;
+FETCH FORWARD 1 pgss_cursor;
+FETCH FORWARD 2 pgss_cursor;
+FETCH FORWARD 3 pgss_cursor;
+FETCH FORWARD -1 pgss_cursor;
+-- explicit FETCH BACKWARD
+FETCH BACKWARD pgss_cursor;
+FETCH BACKWARD 1 pgss_cursor;
+FETCH BACKWARD 2 pgss_cursor;
+FETCH BACKWARD 3 pgss_cursor;
+FETCH BACKWARD -1 pgss_cursor;
+COMMIT;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
+
-- Special cases. Keep these ones at the end to avoid conflicts.
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
SET SCHEMA 'foo';
SET SCHEMA 'public';
RESET ALL;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0b5652071d1..0dfcdbfb310 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -7479,6 +7479,8 @@ fetch_args: cursor_name
n->portalname = $1;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_NONE;
$$ = (Node *) n;
}
| from_in cursor_name
@@ -7488,6 +7490,19 @@ fetch_args: cursor_name
n->portalname = $2;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_NONE;
+ $$ = (Node *) n;
+ }
+ | SignedIconst opt_from_in cursor_name
+ {
+ FetchStmt *n = makeNode(FetchStmt);
+
+ n->portalname = $3;
+ n->direction = FETCH_FORWARD;
+ n->howMany = $1;
+ n->location = @1;
+ n->direction_keyword = FETCH_KEYWORD_NONE;
$$ = (Node *) n;
}
| NEXT opt_from_in cursor_name
@@ -7497,6 +7512,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_NEXT;
$$ = (Node *) n;
}
| PRIOR opt_from_in cursor_name
@@ -7506,6 +7523,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_PRIOR;
$$ = (Node *) n;
}
| FIRST_P opt_from_in cursor_name
@@ -7515,6 +7534,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_FIRST;
$$ = (Node *) n;
}
| LAST_P opt_from_in cursor_name
@@ -7524,6 +7545,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = -1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_LAST;
$$ = (Node *) n;
}
| ABSOLUTE_P SignedIconst opt_from_in cursor_name
@@ -7533,6 +7556,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_ABSOLUTE;
n->howMany = $2;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_ABSOLUTE;
$$ = (Node *) n;
}
| RELATIVE_P SignedIconst opt_from_in cursor_name
@@ -7542,15 +7567,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_RELATIVE;
n->howMany = $2;
- $$ = (Node *) n;
- }
- | SignedIconst opt_from_in cursor_name
- {
- FetchStmt *n = makeNode(FetchStmt);
-
- n->portalname = $3;
- n->direction = FETCH_FORWARD;
- n->howMany = $1;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_RELATIVE;
$$ = (Node *) n;
}
| ALL opt_from_in cursor_name
@@ -7560,6 +7578,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_ALL;
$$ = (Node *) n;
}
| FORWARD opt_from_in cursor_name
@@ -7569,6 +7589,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_FORWARD;
$$ = (Node *) n;
}
| FORWARD SignedIconst opt_from_in cursor_name
@@ -7578,6 +7600,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = $2;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_FORWARD;
$$ = (Node *) n;
}
| FORWARD ALL opt_from_in cursor_name
@@ -7587,6 +7611,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_FORWARD_ALL;
$$ = (Node *) n;
}
| BACKWARD opt_from_in cursor_name
@@ -7596,6 +7622,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_BACKWARD;
$$ = (Node *) n;
}
| BACKWARD SignedIconst opt_from_in cursor_name
@@ -7605,6 +7633,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = $2;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_BACKWARD;
$$ = (Node *) n;
}
| BACKWARD ALL opt_from_in cursor_name
@@ -7614,6 +7644,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_BACKWARD_ALL;
$$ = (Node *) n;
}
;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index dd00ab420b8..9fb050c0705 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3422,15 +3422,41 @@ typedef enum FetchDirection
FETCH_RELATIVE,
} FetchDirection;
+typedef enum FetchDirectionKeywords
+{
+ FETCH_KEYWORD_NONE,
+ FETCH_KEYWORD_NEXT,
+ FETCH_KEYWORD_PRIOR,
+ FETCH_KEYWORD_FIRST,
+ FETCH_KEYWORD_LAST,
+ FETCH_KEYWORD_ABSOLUTE,
+ FETCH_KEYWORD_RELATIVE,
+ FETCH_KEYWORD_ALL,
+ FETCH_KEYWORD_FORWARD,
+ FETCH_KEYWORD_FORWARD_ALL,
+ FETCH_KEYWORD_BACKWARD,
+ FETCH_KEYWORD_BACKWARD_ALL,
+} FetchDirectionKeywords;
+
#define FETCH_ALL LONG_MAX
typedef struct FetchStmt
{
NodeTag type;
FetchDirection direction; /* see above */
- long howMany; /* number of rows, or position argument */
+ /* number of rows, or position argument */
+ long howMany pg_node_attr(query_jumble_ignore);
char *portalname; /* name of portal (cursor) */
bool ismove; /* true if MOVE */
+
+ /*
+ * If an direction_keyword (i.e., FETCH FORWARD) is used, set this field
+ * to distinguish it from its numeric counterpart (i.e., FETCH 1). This
+ * value is set only within gram.y.
+ */
+ FetchDirectionKeywords direction_keyword;
+
+ ParseLoc location pg_node_attr(query_jumble_location);
} FetchStmt;
/* ----------------------
--
2.43.0
On Wed, Jun 04, 2025 at 11:51:56AM -0500, Sami Imseih wrote:
An enum would shine here IMO, because the value could be
self-documented and one would not need to guess what each integer
value means. That could be something like a direction_keyword, filled
with FETCH_KEYWORD_LAST, FETCH_KEYWORD_BACKWARD, etc.Done in v4.
Mostly good here, some more comments.
+ * If an direction_keyword (i.e., FETCH FORWARD) is used, set this field
+ * to distinguish it from its numeric counterpart (i.e., FETCH 1). This
+ * value is set only within gram.y.
One nitpick comment here is that I would have mentioned that this
matters for query jumbling.
The tests can be in cursors.sql and not utility.sql, which is the test
area for... Cursors. :D
FetchDirectionKeywords could be in typedefs.list to avoid the
indentation accident with the structure definition. There's no actual
practice about that for committers, this note will just serve as a
self-reminder once the v19 branch opens for business when I handle
this patch.
--
Michael
+ * If an direction_keyword (i.e., FETCH FORWARD) is used, set this field + * to distinguish it from its numeric counterpart (i.e., FETCH 1). This + * value is set only within gram.y.One nitpick comment here is that I would have mentioned that this
matters for query jumbling.
Done
The tests can be in cursors.sql and not utility.sql, which is the test
area for... Cursors. :D
Yeah, not sure how I missed the cursor.sql file.
FetchDirectionKeywords could be in typedefs.list to avoid the
indentation accident with the structure definition.
Done.
v5 attached.
--
Sami
Attachments:
v5-0001-Normalize-variable-fetch-sizes-in-a-FETCH-command.patchapplication/octet-stream; name=v5-0001-Normalize-variable-fetch-sizes-in-a-FETCH-command.patchDownload
From 92d72e1ad725faa3fc5675e3ac8278a03b02919f Mon Sep 17 00:00:00 2001
From: Ubuntu <ubuntu@ip-172-31-38-230.ec2.internal>
Date: Tue, 3 Jun 2025 17:58:19 +0000
Subject: [PATCH v5 1/1] Normalize variable fetch sizes in a FETCH command
Prior to this patch, every FETCH call would generate a unique queryId.
This led to significant bloat in pg_stat_statements, as repeatedly calling
a specific cursor would result in a new queryId each time. For example,
FETCH 1 c1; and FETCH 2 c1; would produce different queryIds.
This patch improves the situation by normalizing the fetch size, so semantically
similar statements generate the same queryId. As a result, statements like the below,
which differ syntactically but have the same effect, will now share a single queryId:
FETCH FROM c1;
FETCH NEXT c1;
FETCH 1 c1;
Discussion: https://www.postgresql.org/message-id/flat/CAA5RZ0tA6LbHCg2qSS%2BKuM850BZC_%2BZgHV7Ug6BXw22TNyF%2BMA%40mail.gmail.com
---
.../pg_stat_statements/expected/cursors.out | 173 +++++++++++++++++-
.../expected/level_tracking.out | 4 +-
.../pg_stat_statements/expected/utility.out | 2 +-
contrib/pg_stat_statements/sql/cursors.sql | 51 ++++++
src/backend/parser/gram.y | 50 ++++-
src/include/nodes/parsenodes.h | 28 ++-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 294 insertions(+), 15 deletions(-)
diff --git a/contrib/pg_stat_statements/expected/cursors.out b/contrib/pg_stat_statements/expected/cursors.out
index 0fc4b2c098d0..afe5b7b52d5b 100644
--- a/contrib/pg_stat_statements/expected/cursors.out
+++ b/contrib/pg_stat_statements/expected/cursors.out
@@ -57,8 +57,8 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 0 | COMMIT
1 | 0 | DECLARE cursor_stats_1 CURSOR WITH HOLD FOR SELECT $1
1 | 0 | DECLARE cursor_stats_2 CURSOR WITH HOLD FOR SELECT $1
- 1 | 1 | FETCH 1 IN cursor_stats_1
- 1 | 1 | FETCH 1 IN cursor_stats_2
+ 1 | 1 | FETCH $1 IN cursor_stats_1
+ 1 | 1 | FETCH $1 IN cursor_stats_2
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(9 rows)
@@ -68,3 +68,172 @@ SELECT pg_stat_statements_reset() IS NOT NULL AS t;
t
(1 row)
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+-- Normalization of FETCH statements
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+-- implicit directions
+FETCH pgss_cursor;
+--
+(1 row)
+
+FETCH 1 pgss_cursor;
+--
+(1 row)
+
+FETCH 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH 3 pgss_cursor;
+--
+(3 rows)
+
+FETCH -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit NEXT
+FETCH NEXT pgss_cursor;
+--
+(1 row)
+
+-- explicit PRIOR
+FETCH PRIOR pgss_cursor;
+--
+(1 row)
+
+-- explicit FIRST
+FETCH FIRST pgss_cursor;
+--
+(1 row)
+
+-- explicit LAST
+FETCH LAST pgss_cursor;
+--
+(1 row)
+
+-- explicit ABSOLUTE
+FETCH ABSOLUTE 1 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 2 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 3 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit RELATIVE
+FETCH RELATIVE 1 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE 2 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE 3 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit FORWARD
+FETCH ALL pgss_cursor;
+--
+(0 rows)
+
+-- explicit FORWARD ALL
+FETCH FORWARD ALL pgss_cursor;
+--
+(0 rows)
+
+-- explicit BACKWARD ALL
+FETCH BACKWARD ALL pgss_cursor;
+--
+(50 rows)
+
+-- explicit FETCH FORWARD
+FETCH FORWARD pgss_cursor;
+--
+(1 row)
+
+FETCH FORWARD 1 pgss_cursor;
+--
+(1 row)
+
+FETCH FORWARD 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH FORWARD 3 pgss_cursor;
+--
+(3 rows)
+
+FETCH FORWARD -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit FETCH BACKWARD
+FETCH BACKWARD pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD 1 pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH BACKWARD 3 pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD -1 pgss_cursor;
+--
+(1 row)
+
+COMMIT;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
+ calls | query
+-------+--------------------------------------------------------------------
+ 1 | BEGIN
+ 1 | COMMIT
+ 1 | DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series($1, $2)
+ 4 | FETCH ABSOLUTE $1 pgss_cursor
+ 1 | FETCH ALL pgss_cursor
+ 1 | FETCH BACKWARD ALL pgss_cursor
+ 5 | FETCH BACKWARD pgss_cursor
+ 1 | FETCH FIRST pgss_cursor
+ 1 | FETCH FORWARD ALL pgss_cursor
+ 5 | FETCH FORWARD pgss_cursor
+ 1 | FETCH LAST pgss_cursor
+ 1 | FETCH NEXT pgss_cursor
+ 1 | FETCH PRIOR pgss_cursor
+ 4 | FETCH RELATIVE $1 pgss_cursor
+ 5 | FETCH pgss_cursor
+ 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
+(16 rows)
+
diff --git a/contrib/pg_stat_statements/expected/level_tracking.out b/contrib/pg_stat_statements/expected/level_tracking.out
index 75e785e1719e..ce4bd3a823d8 100644
--- a/contrib/pg_stat_statements/expected/level_tracking.out
+++ b/contrib/pg_stat_statements/expected/level_tracking.out
@@ -1142,7 +1142,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | CLOSE foocur
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * from stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | FETCH FORWARD $1 FROM foocur
f | 1 | SELECT * from stats_track_tab
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(7 rows)
@@ -1172,7 +1172,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | CLOSE foocur
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * FROM stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | FETCH FORWARD $1 FROM foocur
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(6 rows)
diff --git a/contrib/pg_stat_statements/expected/utility.out b/contrib/pg_stat_statements/expected/utility.out
index aa4f0f7e6280..9a413505f952 100644
--- a/contrib/pg_stat_statements/expected/utility.out
+++ b/contrib/pg_stat_statements/expected/utility.out
@@ -702,7 +702,7 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 13 | CREATE MATERIALIZED VIEW pgss_matv AS SELECT * FROM pgss_ctas
1 | 10 | CREATE TABLE pgss_ctas AS SELECT a, $1 b FROM generate_series($2, $3) a
1 | 0 | DECLARE pgss_cursor CURSOR FOR SELECT * FROM pgss_matv
- 1 | 5 | FETCH FORWARD 5 pgss_cursor
+ 1 | 5 | FETCH FORWARD $1 pgss_cursor
1 | 7 | FETCH FORWARD ALL pgss_cursor
1 | 1 | FETCH NEXT pgss_cursor
1 | 13 | REFRESH MATERIALIZED VIEW pgss_matv
diff --git a/contrib/pg_stat_statements/sql/cursors.sql b/contrib/pg_stat_statements/sql/cursors.sql
index 61738ac470e8..68ae91882293 100644
--- a/contrib/pg_stat_statements/sql/cursors.sql
+++ b/contrib/pg_stat_statements/sql/cursors.sql
@@ -28,3 +28,54 @@ COMMIT;
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+
+-- Normalization of FETCH statements
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+-- implicit directions
+FETCH pgss_cursor;
+FETCH 1 pgss_cursor;
+FETCH 2 pgss_cursor;
+FETCH 3 pgss_cursor;
+FETCH -1 pgss_cursor;
+-- explicit NEXT
+FETCH NEXT pgss_cursor;
+-- explicit PRIOR
+FETCH PRIOR pgss_cursor;
+-- explicit FIRST
+FETCH FIRST pgss_cursor;
+-- explicit LAST
+FETCH LAST pgss_cursor;
+-- explicit ABSOLUTE
+FETCH ABSOLUTE 1 pgss_cursor;
+FETCH ABSOLUTE 2 pgss_cursor;
+FETCH ABSOLUTE 3 pgss_cursor;
+FETCH ABSOLUTE -1 pgss_cursor;
+-- explicit RELATIVE
+FETCH RELATIVE 1 pgss_cursor;
+FETCH RELATIVE 2 pgss_cursor;
+FETCH RELATIVE 3 pgss_cursor;
+FETCH RELATIVE -1 pgss_cursor;
+-- explicit FORWARD
+FETCH ALL pgss_cursor;
+-- explicit FORWARD ALL
+FETCH FORWARD ALL pgss_cursor;
+-- explicit BACKWARD ALL
+FETCH BACKWARD ALL pgss_cursor;
+-- explicit FETCH FORWARD
+FETCH FORWARD pgss_cursor;
+FETCH FORWARD 1 pgss_cursor;
+FETCH FORWARD 2 pgss_cursor;
+FETCH FORWARD 3 pgss_cursor;
+FETCH FORWARD -1 pgss_cursor;
+-- explicit FETCH BACKWARD
+FETCH BACKWARD pgss_cursor;
+FETCH BACKWARD 1 pgss_cursor;
+FETCH BACKWARD 2 pgss_cursor;
+FETCH BACKWARD 3 pgss_cursor;
+FETCH BACKWARD -1 pgss_cursor;
+COMMIT;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0b5652071d11..0dfcdbfb3100 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -7479,6 +7479,8 @@ fetch_args: cursor_name
n->portalname = $1;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_NONE;
$$ = (Node *) n;
}
| from_in cursor_name
@@ -7488,6 +7490,19 @@ fetch_args: cursor_name
n->portalname = $2;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_NONE;
+ $$ = (Node *) n;
+ }
+ | SignedIconst opt_from_in cursor_name
+ {
+ FetchStmt *n = makeNode(FetchStmt);
+
+ n->portalname = $3;
+ n->direction = FETCH_FORWARD;
+ n->howMany = $1;
+ n->location = @1;
+ n->direction_keyword = FETCH_KEYWORD_NONE;
$$ = (Node *) n;
}
| NEXT opt_from_in cursor_name
@@ -7497,6 +7512,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_NEXT;
$$ = (Node *) n;
}
| PRIOR opt_from_in cursor_name
@@ -7506,6 +7523,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_PRIOR;
$$ = (Node *) n;
}
| FIRST_P opt_from_in cursor_name
@@ -7515,6 +7534,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_FIRST;
$$ = (Node *) n;
}
| LAST_P opt_from_in cursor_name
@@ -7524,6 +7545,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = -1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_LAST;
$$ = (Node *) n;
}
| ABSOLUTE_P SignedIconst opt_from_in cursor_name
@@ -7533,6 +7556,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_ABSOLUTE;
n->howMany = $2;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_ABSOLUTE;
$$ = (Node *) n;
}
| RELATIVE_P SignedIconst opt_from_in cursor_name
@@ -7542,15 +7567,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_RELATIVE;
n->howMany = $2;
- $$ = (Node *) n;
- }
- | SignedIconst opt_from_in cursor_name
- {
- FetchStmt *n = makeNode(FetchStmt);
-
- n->portalname = $3;
- n->direction = FETCH_FORWARD;
- n->howMany = $1;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_RELATIVE;
$$ = (Node *) n;
}
| ALL opt_from_in cursor_name
@@ -7560,6 +7578,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_ALL;
$$ = (Node *) n;
}
| FORWARD opt_from_in cursor_name
@@ -7569,6 +7589,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_FORWARD;
$$ = (Node *) n;
}
| FORWARD SignedIconst opt_from_in cursor_name
@@ -7578,6 +7600,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = $2;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_FORWARD;
$$ = (Node *) n;
}
| FORWARD ALL opt_from_in cursor_name
@@ -7587,6 +7611,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_FORWARD_ALL;
$$ = (Node *) n;
}
| BACKWARD opt_from_in cursor_name
@@ -7596,6 +7622,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_BACKWARD;
$$ = (Node *) n;
}
| BACKWARD SignedIconst opt_from_in cursor_name
@@ -7605,6 +7633,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = $2;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_BACKWARD;
$$ = (Node *) n;
}
| BACKWARD ALL opt_from_in cursor_name
@@ -7614,6 +7644,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_BACKWARD_ALL;
$$ = (Node *) n;
}
;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index dd00ab420b8a..2f464ab23b8e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3422,15 +3422,41 @@ typedef enum FetchDirection
FETCH_RELATIVE,
} FetchDirection;
+typedef enum FetchDirectionKeywords
+{
+ FETCH_KEYWORD_NONE,
+ FETCH_KEYWORD_NEXT,
+ FETCH_KEYWORD_PRIOR,
+ FETCH_KEYWORD_FIRST,
+ FETCH_KEYWORD_LAST,
+ FETCH_KEYWORD_ABSOLUTE,
+ FETCH_KEYWORD_RELATIVE,
+ FETCH_KEYWORD_ALL,
+ FETCH_KEYWORD_FORWARD,
+ FETCH_KEYWORD_FORWARD_ALL,
+ FETCH_KEYWORD_BACKWARD,
+ FETCH_KEYWORD_BACKWARD_ALL,
+} FetchDirectionKeywords;
+
#define FETCH_ALL LONG_MAX
typedef struct FetchStmt
{
NodeTag type;
FetchDirection direction; /* see above */
- long howMany; /* number of rows, or position argument */
+ /* number of rows, or position argument */
+ long howMany pg_node_attr(query_jumble_ignore);
char *portalname; /* name of portal (cursor) */
bool ismove; /* true if MOVE */
+
+ /*
+ * This field is set in gram.y when a direction_keyword (e.g., FETCH
+ * FORWARD) is used, to distinguish it from a numeric variant (e.g., FETCH
+ * 1) for the purpose of query jumbling.
+ */
+ FetchDirectionKeywords direction_keyword;
+
+ ParseLoc location pg_node_attr(query_jumble_location);
} FetchStmt;
/* ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a8346cda633a..8b15ac5c307e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -4305,3 +4305,4 @@ zic_t
ExplainExtensionOption
ExplainOptionHandler
overexplain_options
+FetchDirectionKeywords
--
2.39.5 (Apple Git-154)
On Thu, Jun 05, 2025 at 07:42:48AM -0500, Sami Imseih wrote:
v5 attached.
I'm OK with this version, so switching that as ready for committer.
--
Michael
rebased patch.
Regards,
Sami
Attachments:
v6-0001-Normalize-variable-fetch-sizes-in-a-FETCH-command.patchapplication/octet-stream; name=v6-0001-Normalize-variable-fetch-sizes-in-a-FETCH-command.patchDownload
From f7cad0272892a1ce84a97edff74804199aebc888 Mon Sep 17 00:00:00 2001
From: Ubuntu <ubuntu@ip-172-31-38-230.ec2.internal>
Date: Tue, 3 Jun 2025 17:58:19 +0000
Subject: [PATCH v6 1/1] Normalize variable fetch sizes in a FETCH command
Prior to this patch, every FETCH call would generate a unique queryId.
This led to significant bloat in pg_stat_statements, as repeatedly calling
a specific cursor would result in a new queryId each time. For example,
FETCH 1 c1; and FETCH 2 c1; would produce different queryIds.
This patch improves the situation by normalizing the fetch size, so semantically
similar statements generate the same queryId. As a result, statements like the below,
which differ syntactically but have the same effect, will now share a single queryId:
FETCH FROM c1;
FETCH NEXT c1;
FETCH 1 c1;
Discussion: https://www.postgresql.org/message-id/flat/CAA5RZ0tA6LbHCg2qSS%2BKuM850BZC_%2BZgHV7Ug6BXw22TNyF%2BMA%40mail.gmail.com
---
.../pg_stat_statements/expected/cursors.out | 173 +++++++++++++++++-
.../expected/level_tracking.out | 4 +-
.../pg_stat_statements/expected/utility.out | 2 +-
contrib/pg_stat_statements/sql/cursors.sql | 51 ++++++
src/backend/parser/gram.y | 50 ++++-
src/include/nodes/parsenodes.h | 28 ++-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 294 insertions(+), 15 deletions(-)
diff --git a/contrib/pg_stat_statements/expected/cursors.out b/contrib/pg_stat_statements/expected/cursors.out
index 0fc4b2c098d..afe5b7b52d5 100644
--- a/contrib/pg_stat_statements/expected/cursors.out
+++ b/contrib/pg_stat_statements/expected/cursors.out
@@ -57,8 +57,8 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 0 | COMMIT
1 | 0 | DECLARE cursor_stats_1 CURSOR WITH HOLD FOR SELECT $1
1 | 0 | DECLARE cursor_stats_2 CURSOR WITH HOLD FOR SELECT $1
- 1 | 1 | FETCH 1 IN cursor_stats_1
- 1 | 1 | FETCH 1 IN cursor_stats_2
+ 1 | 1 | FETCH $1 IN cursor_stats_1
+ 1 | 1 | FETCH $1 IN cursor_stats_2
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(9 rows)
@@ -68,3 +68,172 @@ SELECT pg_stat_statements_reset() IS NOT NULL AS t;
t
(1 row)
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+-- Normalization of FETCH statements
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+-- implicit directions
+FETCH pgss_cursor;
+--
+(1 row)
+
+FETCH 1 pgss_cursor;
+--
+(1 row)
+
+FETCH 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH 3 pgss_cursor;
+--
+(3 rows)
+
+FETCH -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit NEXT
+FETCH NEXT pgss_cursor;
+--
+(1 row)
+
+-- explicit PRIOR
+FETCH PRIOR pgss_cursor;
+--
+(1 row)
+
+-- explicit FIRST
+FETCH FIRST pgss_cursor;
+--
+(1 row)
+
+-- explicit LAST
+FETCH LAST pgss_cursor;
+--
+(1 row)
+
+-- explicit ABSOLUTE
+FETCH ABSOLUTE 1 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 2 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE 3 pgss_cursor;
+--
+(1 row)
+
+FETCH ABSOLUTE -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit RELATIVE
+FETCH RELATIVE 1 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE 2 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE 3 pgss_cursor;
+--
+(0 rows)
+
+FETCH RELATIVE -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit FORWARD
+FETCH ALL pgss_cursor;
+--
+(0 rows)
+
+-- explicit FORWARD ALL
+FETCH FORWARD ALL pgss_cursor;
+--
+(0 rows)
+
+-- explicit BACKWARD ALL
+FETCH BACKWARD ALL pgss_cursor;
+--
+(50 rows)
+
+-- explicit FETCH FORWARD
+FETCH FORWARD pgss_cursor;
+--
+(1 row)
+
+FETCH FORWARD 1 pgss_cursor;
+--
+(1 row)
+
+FETCH FORWARD 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH FORWARD 3 pgss_cursor;
+--
+(3 rows)
+
+FETCH FORWARD -1 pgss_cursor;
+--
+(1 row)
+
+-- explicit FETCH BACKWARD
+FETCH BACKWARD pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD 1 pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD 2 pgss_cursor;
+--
+(2 rows)
+
+FETCH BACKWARD 3 pgss_cursor;
+--
+(1 row)
+
+FETCH BACKWARD -1 pgss_cursor;
+--
+(1 row)
+
+COMMIT;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
+ calls | query
+-------+--------------------------------------------------------------------
+ 1 | BEGIN
+ 1 | COMMIT
+ 1 | DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series($1, $2)
+ 4 | FETCH ABSOLUTE $1 pgss_cursor
+ 1 | FETCH ALL pgss_cursor
+ 1 | FETCH BACKWARD ALL pgss_cursor
+ 5 | FETCH BACKWARD pgss_cursor
+ 1 | FETCH FIRST pgss_cursor
+ 1 | FETCH FORWARD ALL pgss_cursor
+ 5 | FETCH FORWARD pgss_cursor
+ 1 | FETCH LAST pgss_cursor
+ 1 | FETCH NEXT pgss_cursor
+ 1 | FETCH PRIOR pgss_cursor
+ 4 | FETCH RELATIVE $1 pgss_cursor
+ 5 | FETCH pgss_cursor
+ 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
+(16 rows)
+
diff --git a/contrib/pg_stat_statements/expected/level_tracking.out b/contrib/pg_stat_statements/expected/level_tracking.out
index 8213fcd2e61..8e8388dd5cb 100644
--- a/contrib/pg_stat_statements/expected/level_tracking.out
+++ b/contrib/pg_stat_statements/expected/level_tracking.out
@@ -1147,7 +1147,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * from stats_track_tab
f | 1 | DECLARE FOOCUR CURSOR FOR SELECT * from stats_track_tab;
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | FETCH FORWARD $1 FROM foocur
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(7 rows)
@@ -1176,7 +1176,7 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | CLOSE foocur
t | 1 | COMMIT
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * FROM stats_track_tab
- t | 1 | FETCH FORWARD 1 FROM foocur
+ t | 1 | FETCH FORWARD $1 FROM foocur
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
(6 rows)
diff --git a/contrib/pg_stat_statements/expected/utility.out b/contrib/pg_stat_statements/expected/utility.out
index 060d4416dd7..e4d6564ea5b 100644
--- a/contrib/pg_stat_statements/expected/utility.out
+++ b/contrib/pg_stat_statements/expected/utility.out
@@ -702,7 +702,7 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 13 | CREATE MATERIALIZED VIEW pgss_matv AS SELECT * FROM pgss_ctas
1 | 10 | CREATE TABLE pgss_ctas AS SELECT a, $1 b FROM generate_series($2, $3) a
1 | 0 | DECLARE pgss_cursor CURSOR FOR SELECT * FROM pgss_matv
- 1 | 5 | FETCH FORWARD 5 pgss_cursor
+ 1 | 5 | FETCH FORWARD $1 pgss_cursor
1 | 7 | FETCH FORWARD ALL pgss_cursor
1 | 1 | FETCH NEXT pgss_cursor
1 | 13 | REFRESH MATERIALIZED VIEW pgss_matv
diff --git a/contrib/pg_stat_statements/sql/cursors.sql b/contrib/pg_stat_statements/sql/cursors.sql
index 61738ac470e..68ae9188229 100644
--- a/contrib/pg_stat_statements/sql/cursors.sql
+++ b/contrib/pg_stat_statements/sql/cursors.sql
@@ -28,3 +28,54 @@ COMMIT;
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+
+-- Normalization of FETCH statements
+SELECT pg_stat_statements_reset() IS NOT NULL AS t;
+BEGIN;
+DECLARE pgss_cursor CURSOR FOR SELECT FROM generate_series(1, 50);
+-- implicit directions
+FETCH pgss_cursor;
+FETCH 1 pgss_cursor;
+FETCH 2 pgss_cursor;
+FETCH 3 pgss_cursor;
+FETCH -1 pgss_cursor;
+-- explicit NEXT
+FETCH NEXT pgss_cursor;
+-- explicit PRIOR
+FETCH PRIOR pgss_cursor;
+-- explicit FIRST
+FETCH FIRST pgss_cursor;
+-- explicit LAST
+FETCH LAST pgss_cursor;
+-- explicit ABSOLUTE
+FETCH ABSOLUTE 1 pgss_cursor;
+FETCH ABSOLUTE 2 pgss_cursor;
+FETCH ABSOLUTE 3 pgss_cursor;
+FETCH ABSOLUTE -1 pgss_cursor;
+-- explicit RELATIVE
+FETCH RELATIVE 1 pgss_cursor;
+FETCH RELATIVE 2 pgss_cursor;
+FETCH RELATIVE 3 pgss_cursor;
+FETCH RELATIVE -1 pgss_cursor;
+-- explicit FORWARD
+FETCH ALL pgss_cursor;
+-- explicit FORWARD ALL
+FETCH FORWARD ALL pgss_cursor;
+-- explicit BACKWARD ALL
+FETCH BACKWARD ALL pgss_cursor;
+-- explicit FETCH FORWARD
+FETCH FORWARD pgss_cursor;
+FETCH FORWARD 1 pgss_cursor;
+FETCH FORWARD 2 pgss_cursor;
+FETCH FORWARD 3 pgss_cursor;
+FETCH FORWARD -1 pgss_cursor;
+-- explicit FETCH BACKWARD
+FETCH BACKWARD pgss_cursor;
+FETCH BACKWARD 1 pgss_cursor;
+FETCH BACKWARD 2 pgss_cursor;
+FETCH BACKWARD 3 pgss_cursor;
+FETCH BACKWARD -1 pgss_cursor;
+COMMIT;
+SELECT calls, query FROM pg_stat_statements ORDER BY query COLLATE "C";
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 50f53159d58..1c11b235aa6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -7477,6 +7477,8 @@ fetch_args: cursor_name
n->portalname = $1;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_NONE;
$$ = (Node *) n;
}
| from_in cursor_name
@@ -7486,6 +7488,19 @@ fetch_args: cursor_name
n->portalname = $2;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_NONE;
+ $$ = (Node *) n;
+ }
+ | SignedIconst opt_from_in cursor_name
+ {
+ FetchStmt *n = makeNode(FetchStmt);
+
+ n->portalname = $3;
+ n->direction = FETCH_FORWARD;
+ n->howMany = $1;
+ n->location = @1;
+ n->direction_keyword = FETCH_KEYWORD_NONE;
$$ = (Node *) n;
}
| NEXT opt_from_in cursor_name
@@ -7495,6 +7510,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_NEXT;
$$ = (Node *) n;
}
| PRIOR opt_from_in cursor_name
@@ -7504,6 +7521,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_PRIOR;
$$ = (Node *) n;
}
| FIRST_P opt_from_in cursor_name
@@ -7513,6 +7532,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_FIRST;
$$ = (Node *) n;
}
| LAST_P opt_from_in cursor_name
@@ -7522,6 +7543,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_ABSOLUTE;
n->howMany = -1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_LAST;
$$ = (Node *) n;
}
| ABSOLUTE_P SignedIconst opt_from_in cursor_name
@@ -7531,6 +7554,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_ABSOLUTE;
n->howMany = $2;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_ABSOLUTE;
$$ = (Node *) n;
}
| RELATIVE_P SignedIconst opt_from_in cursor_name
@@ -7540,15 +7565,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_RELATIVE;
n->howMany = $2;
- $$ = (Node *) n;
- }
- | SignedIconst opt_from_in cursor_name
- {
- FetchStmt *n = makeNode(FetchStmt);
-
- n->portalname = $3;
- n->direction = FETCH_FORWARD;
- n->howMany = $1;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_RELATIVE;
$$ = (Node *) n;
}
| ALL opt_from_in cursor_name
@@ -7558,6 +7576,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_ALL;
$$ = (Node *) n;
}
| FORWARD opt_from_in cursor_name
@@ -7567,6 +7587,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_FORWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_FORWARD;
$$ = (Node *) n;
}
| FORWARD SignedIconst opt_from_in cursor_name
@@ -7576,6 +7598,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = $2;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_FORWARD;
$$ = (Node *) n;
}
| FORWARD ALL opt_from_in cursor_name
@@ -7585,6 +7609,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_FORWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_FORWARD_ALL;
$$ = (Node *) n;
}
| BACKWARD opt_from_in cursor_name
@@ -7594,6 +7620,8 @@ fetch_args: cursor_name
n->portalname = $3;
n->direction = FETCH_BACKWARD;
n->howMany = 1;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_BACKWARD;
$$ = (Node *) n;
}
| BACKWARD SignedIconst opt_from_in cursor_name
@@ -7603,6 +7631,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = $2;
+ n->location = @2;
+ n->direction_keyword = FETCH_KEYWORD_BACKWARD;
$$ = (Node *) n;
}
| BACKWARD ALL opt_from_in cursor_name
@@ -7612,6 +7642,8 @@ fetch_args: cursor_name
n->portalname = $4;
n->direction = FETCH_BACKWARD;
n->howMany = FETCH_ALL;
+ n->location = -1;
+ n->direction_keyword = FETCH_KEYWORD_BACKWARD_ALL;
$$ = (Node *) n;
}
;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ba12678d1cb..2a83b9c3074 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -3422,15 +3422,41 @@ typedef enum FetchDirection
FETCH_RELATIVE,
} FetchDirection;
+typedef enum FetchDirectionKeywords
+{
+ FETCH_KEYWORD_NONE,
+ FETCH_KEYWORD_NEXT,
+ FETCH_KEYWORD_PRIOR,
+ FETCH_KEYWORD_FIRST,
+ FETCH_KEYWORD_LAST,
+ FETCH_KEYWORD_ABSOLUTE,
+ FETCH_KEYWORD_RELATIVE,
+ FETCH_KEYWORD_ALL,
+ FETCH_KEYWORD_FORWARD,
+ FETCH_KEYWORD_FORWARD_ALL,
+ FETCH_KEYWORD_BACKWARD,
+ FETCH_KEYWORD_BACKWARD_ALL,
+} FetchDirectionKeywords;
+
#define FETCH_ALL LONG_MAX
typedef struct FetchStmt
{
NodeTag type;
FetchDirection direction; /* see above */
- long howMany; /* number of rows, or position argument */
+ /* number of rows, or position argument */
+ long howMany pg_node_attr(query_jumble_ignore);
char *portalname; /* name of portal (cursor) */
bool ismove; /* true if MOVE */
+
+ /*
+ * This field is set in gram.y when a direction_keyword (e.g., FETCH
+ * FORWARD) is used, to distinguish it from a numeric variant (e.g., FETCH
+ * 1) for the purpose of query jumbling.
+ */
+ FetchDirectionKeywords direction_keyword;
+
+ ParseLoc location pg_node_attr(query_jumble_location);
} FetchStmt;
/* ----------------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 32d6e718adc..897a061d25d 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -805,6 +805,7 @@ FastPathStrongRelationLockData
FdwInfo
FdwRoutine
FetchDirection
+FetchDirectionKeywords
FetchStmt
FieldSelect
FieldStore
--
2.43.0
On Mon, Jun 30, 2025 at 03:37:08PM +0300, Sami Imseih wrote:
rebased patch.
There were two extra pg_stat_statements_reset() calls added to
cursors.sql that were not necessary. I have also cut by one the
number of FETCH queries with the numbers, as it does not change the
coverage outcome, tweaked a bit the comments, and the result looked
fine. At the end, applied.
--
Michael