回复:[Internet]Re: [PATCH] Prevent replacement of a function if it's used in an index expression and is not IMMUTABLE

Started by sundayjiang(蒋浩天)6 months ago5 messages
#1sundayjiang(蒋浩天)
sundayjiang@tencent.com
1 attachment(s)

Hi hackers,

Thanks for the valuable feedback on my previous patch.

Based on the review comments, I have updated the patch as follows:

- Limited the error trigger only to the case where the original function is IMMUTABLE but the new definition is not.
- Changed the error code to ERRCODE_FEATURE_NOT_SUPPORTED for better semantics.
- Improved code indentation and style to align with project conventions.
- Added a regression test case in create_function_sql.sql to verify the new behavior.

The full updated patch is attached below.

Please take a look at this revised version. I look forward to your feedback and suggestions.

Best regards,  
xiaojiluo (Tencent Yunding Lab)

jian he<jian.universality@gmail.com&gt;&nbsp;在 2025年7月9日 周三 10:34 写道:

On Mon, Jun 30, 2025 at 5:34 PM sundayjiang(蒋浩天)
<sundayjiang@tencent.com&gt; wrote:
&gt;
&gt; The purpose of this patch is to prevent replacing a function via `CREATE OR REPLACE FUNCTION` with a new definition that is not marked as `IMMUTABLE`, if the existing function is referenced by an index expression.
&gt;
&gt; Replacing such functions may lead to index corruption or runtime semantic inconsistencies, especially when the function’s output is not stable for the same input.
&gt;
&gt; If a function is used in an index, it can only be replaced if it is declared as `IMMUTABLE`.
&gt;

looking at the right above code ``if (oldproc-&gt;prokind != prokind)``

+ if(oldproc-&gt;prokind == PROKIND_FUNCTION &amp;&amp; volatility !=
PROVOLATILE_IMMUTABLE){

we can change it to

+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if(prokind == PROKIND_FUNCTION &amp;&amp; oldproc-&gt;provolatile ==
PROVOLATILE_IMMUTABLE &amp;&amp;
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; volatility != PROVOLATILE_IMMUTABLE)
&nbsp;+&nbsp;&nbsp;&nbsp;&nbsp; {
curly brace generally begins with a new line.
if (index_found)
+ ereport(ERROR,
+ (errcode(ERRCODE_DEPENDENT_OBJECTS_STILL_EXIST),
+ errmsg("cannot replace function \"%s\" with a non-IMMUTABLE function
because it is used by an index",
+ procedureName)));
Here, errcode ERRCODE_FEATURE_NOT_SUPPORTED would be more appropriate.

you can add a simple test in src/test/regress/sql/create_function_sql.sql
for example:
CREATE OR REPLACE FUNCTION get_a_int(int default 2) RETURNS int
IMMUTABLE AS 'select $1' LANGUAGE sql;
create table t1(a int);
create index on t1((get_a_int(a)));
CREATE OR REPLACE FUNCTION get_a_int(int default 2) RETURNS int AS
'select $1' LANGUAGE sql;

Attachments:

v2-0001-Prevent-replacement-of-a-function-if-it-s-used-in-an.patchapplication/octet-stream; charset=utf-8; name=v2-0001-Prevent-replacement-of-a-function-if-it-s-used-in-an.patchDownload
From 0970a04a012686ddba64c4ebd38295656dbaada2 Mon Sep 17 00:00:00 2001
From: xiaojiluo <xiaojiluo@tencent.com>
Date: Wed, 25 Jun 2025 12:04:05 +0800
Subject: [PATCH v2] Prevent replacement of a function if it's used in an index
 expression and is not IMMUTABLE

In ProcedureCreate(), add a check to disallow replacing an existing function
if it is referenced by an index expression and is not marked as IMMUTABLE.
Replacing such a function could break index semantics or lead to inconsistent
behavior at runtime, especially if the function's output is not guaranteed
to be stable for the same input.

IMMUTABLE functions are assumed to always return the same output for the same
input and thus are considered safe to replace even when referenced in indexes.

This check adds a dependency scan on pg_depend to detect any usage of the
function in indexes. If the function is found to be in use and is not
IMMUTABLE, an error is thrown.

This refinement ensures safer function replacement behavior by blocking
redefinition only in cases where semantic consistency of indexed expressions
could be compromised.

In v2:
- Limit the error to the case where the original function is IMMUTABLE and
  the new definition is not IMMUTABLE.
- Use errcode FEATURE_NOT_SUPPORTED instead of DEPENDENT_OBJECTS_STILL_EXIST,
  which better reflects the nature of this restriction.
- Adjust indentation and code style according to reviewer suggestions.
- Add a regression test in create_function_sql.sql to verify the behavior.

Author: xiaojiluo <xiaojiluo@tencent.com>
---
 src/backend/catalog/pg_proc.c                 | 57 +++++++++++++++++++
 .../regress/expected/create_function_sql.out  | 17 +++++-
 src/test/regress/sql/create_function_sql.sql  | 13 +++++
 3 files changed, 86 insertions(+), 1 deletion(-)

diff --git a/src/backend/catalog/pg_proc.c b/src/backend/catalog/pg_proc.c
index 5fdcf24d5f8..fd831058c32 100644
--- a/src/backend/catalog/pg_proc.c
+++ b/src/backend/catalog/pg_proc.c
@@ -26,6 +26,8 @@
 #include "catalog/pg_proc.h"
 #include "catalog/pg_transform.h"
 #include "catalog/pg_type.h"
+#include "catalog/pg_depend.h"
+#include "utils/fmgroids.h"
 #include "executor/functions.h"
 #include "funcapi.h"
 #include "mb/pg_wchar.h"
@@ -420,6 +422,61 @@ ProcedureCreate(const char *procedureName,
 					  errdetail("\"%s\" is a window function.", procedureName) :
 					  0)));
 
+		if (oldproc->prokind == PROKIND_FUNCTION &&
+			oldproc->provolatile == PROVOLATILE_IMMUTABLE &&
+			volatility != PROVOLATILE_IMMUTABLE)
+		{
+			Relation depRel = table_open(DependRelationId, AccessShareLock);
+			bool index_found = false;
+			SysScanDesc scan;
+			ScanKeyData key;
+			HeapTuple dtup;
+
+			/* refobjid = oldproc->oid */
+			ScanKeyInit(&key,
+						Anum_pg_depend_refobjid,
+						BTEqualStrategyNumber, F_OIDEQ,
+						ObjectIdGetDatum(oldproc->oid));
+
+			scan = systable_beginscan(depRel,
+									DependReferenceIndexId,
+									true,
+									NULL,
+									1, &key);
+
+			while (HeapTupleIsValid(dtup = systable_getnext(scan)))
+			{
+				Form_pg_depend d = (Form_pg_depend) GETSTRUCT(dtup);
+
+				if (d->classid == RelationRelationId && d->objsubid == 0)
+				{
+					/* query relkind */
+					HeapTuple reltup = SearchSysCache1(RELOID, ObjectIdGetDatum(d->objid));
+					if (HeapTupleIsValid(reltup))
+					{
+						Form_pg_class classForm = (Form_pg_class) GETSTRUCT(reltup);
+						if (classForm->relkind == RELKIND_INDEX)
+						{
+							index_found = true;
+							ReleaseSysCache(reltup);
+							break;
+						}
+						ReleaseSysCache(reltup);
+					}
+				}
+			}
+
+			systable_endscan(scan);
+			table_close(depRel, AccessShareLock);
+
+			if (index_found)
+				ereport(ERROR,
+						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+						errmsg("cannot replace function \"%s\" with a non-IMMUTABLE function because it is used by an index",
+								procedureName)));
+
+		}
+
 		dropcmd = (prokind == PROKIND_PROCEDURE ? "DROP PROCEDURE" :
 				   prokind == PROKIND_AGGREGATE ? "DROP AGGREGATE" :
 				   "DROP FUNCTION");
diff --git a/src/test/regress/expected/create_function_sql.out b/src/test/regress/expected/create_function_sql.out
index 963b6f863ff..6da599e22e2 100644
--- a/src/test/regress/expected/create_function_sql.out
+++ b/src/test/regress/expected/create_function_sql.out
@@ -771,9 +771,22 @@ ERROR:  return type mismatch in function declared to return integer[]
 DETAIL:  Function's final statement must be SELECT or INSERT/UPDATE/DELETE/MERGE RETURNING.
 CONTEXT:  SQL function "test1" during startup
 RESET check_function_bodies;
+-- Test: prevent replacing an IMMUTABLE function used in index with a non-IMMUTABLE one
+CREATE OR REPLACE FUNCTION fidx(int) RETURNS int
+IMMUTABLE LANGUAGE SQL AS $$ SELECT $1 + 1 $$;
+-- Create a table and an index using that function
+CREATE TABLE test_idx(a int);
+CREATE INDEX idx_f ON test_idx((fidx(a)));
+-- This should be allowed
+CREATE OR REPLACE FUNCTION fidx(int) RETURNS int
+IMMUTABLE LANGUAGE SQL AS $$ SELECT $1 + 2 $$;
+-- Try to replace it with a STABLE function (should fail)
+CREATE OR REPLACE FUNCTION fidx(int) RETURNS int
+STABLE LANGUAGE SQL AS $$ SELECT $1 + 1 $$;
+ERROR:  cannot replace function "fidx" with a non-IMMUTABLE function because it is used by an index
 -- Cleanup
 DROP SCHEMA temp_func_test CASCADE;
-NOTICE:  drop cascades to 35 other objects
+NOTICE:  drop cascades to 37 other objects
 DETAIL:  drop cascades to function functest_a_1(text,date)
 drop cascades to function functest_a_2(text[])
 drop cascades to function functest_a_3()
@@ -809,5 +822,7 @@ drop cascades to table ddl_test
 drop cascades to function alter_and_insert()
 drop cascades to function double_append(anyarray,anyelement)
 drop cascades to function test1(anyelement)
+drop cascades to function fidx(integer)
+drop cascades to table test_idx
 DROP USER regress_unpriv_user;
 RESET search_path;
diff --git a/src/test/regress/sql/create_function_sql.sql b/src/test/regress/sql/create_function_sql.sql
index 6d1c102d780..2307eee0160 100644
--- a/src/test/regress/sql/create_function_sql.sql
+++ b/src/test/regress/sql/create_function_sql.sql
@@ -459,6 +459,19 @@ CREATE FUNCTION test1 (anyelement) RETURNS anyarray LANGUAGE SQL
 SELECT test1(0);
 RESET check_function_bodies;
 
+-- Test: prevent replacing an IMMUTABLE function used in index with a non-IMMUTABLE one
+CREATE OR REPLACE FUNCTION fidx(int) RETURNS int
+IMMUTABLE LANGUAGE SQL AS $$ SELECT $1 + 1 $$;
+-- Create a table and an index using that function
+CREATE TABLE test_idx(a int);
+CREATE INDEX idx_f ON test_idx((fidx(a)));
+-- This should be allowed
+CREATE OR REPLACE FUNCTION fidx(int) RETURNS int
+IMMUTABLE LANGUAGE SQL AS $$ SELECT $1 + 2 $$;
+-- Try to replace it with a STABLE function (should fail)
+CREATE OR REPLACE FUNCTION fidx(int) RETURNS int
+STABLE LANGUAGE SQL AS $$ SELECT $1 + 1 $$;
+
 -- Cleanup
 DROP SCHEMA temp_func_test CASCADE;
 DROP USER regress_unpriv_user;
-- 
2.43.0

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: sundayjiang(蒋浩天) (#1)
Re: 回复:[Internet]Re: [PATCH] Prevent replacement of a function if it's used in an index expression and is not IMMUTABLE

"=?utf-8?B?c3VuZGF5amlhbmco6JKL5rWp5aSpKQ==?=" <sundayjiang@tencent.com> writes:

The purpose of this patch is to prevent replacing a function via `CREATE OR REPLACE FUNCTION` with a new definition that is not marked as `IMMUTABLE`, if the existing function is referenced by an index expression.

Replacing such functions may lead to index corruption or runtime semantic inconsistencies, especially when the function’s output is not stable for the same input.

TBH, I find this proposal to be useless nannyism. Replacing a
function that is used in an index is problematic if you change its
behavior (that is, its actual output for given inputs) in any way.
Whether it's marked IMMUTABLE is a very minor side point.

If we could detect whether the new definition results in different
behavior, that would be peachy ... but it's equivalent to solving
Turing's halting problem, so I doubt we're going to get there.

So the only really practical thing we could do about that would be
to forbid CREATE OR REPLACE altogether on functions used in indexes.
That proposal is never going to fly, however.

Another way in which the proposed patch is just a band-aid is that
it doesn't detect indirect dependencies, eg if function f() used
in some index calls function g() and you redefine function g().

So as far as I can see, this patch is slowing down function
redefinition in exchange for not much. We basically have to
trust that people who use user-defined functions in indexes
are aware of the hazard and don't change such functions in
ways that will break their indexes, or at least remember to
REINDEX afterwards. The portion of that hazard that we can
detect automatically is so small that I think we're just
fooling ourselves (and others) to try at all.

regards, tom lane

#3David G. Johnston
david.g.johnston@gmail.com
In reply to: Tom Lane (#2)
Re: 回复:[Internet]Re: [PATCH] Prevent replacement of a function if it's used in an index expression and is not IMMUTABLE

On Wed, Jul 9, 2025 at 9:15 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

"=?utf-8?B?c3VuZGF5amlhbmco6JKL5rWp5aSpKQ==?=" <sundayjiang@tencent.com>
writes:

The purpose of this patch is to prevent replacing a function via

`CREATE OR REPLACE FUNCTION` with a new definition that is not marked as
`IMMUTABLE`, if the existing function is referenced by an index expression.

Replacing such functions may lead to index corruption or runtime

semantic inconsistencies, especially when the function’s output is not
stable for the same input.

TBH, I find this proposal to be useless nannyism. Replacing a
function that is used in an index is problematic if you change its
behavior (that is, its actual output for given inputs) in any way.
Whether it's marked IMMUTABLE is a very minor side point.

Isn't preventing a dump-restore hazard sufficient reason to do this? The
now-volatile function will be dumped as such and the create index during
the restore will fail because of that.

David J.

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: David G. Johnston (#3)
Re: 回复:[Internet]Re: [PATCH] Prevent replacement of a function if it's used in an index expression and is not IMMUTABLE

"David G. Johnston" <david.g.johnston@gmail.com> writes:

On Wed, Jul 9, 2025 at 9:15 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

TBH, I find this proposal to be useless nannyism.

Isn't preventing a dump-restore hazard sufficient reason to do this?

No, I don't think so. If you're not being very careful about revising
functions used in indexes, you are going to have problems a lot sooner
than some future dump/restore cycle.

regards, tom lane

#5David G. Johnston
david.g.johnston@gmail.com
In reply to: Tom Lane (#4)
Re: 回复:[Internet]Re: [PATCH] Prevent replacement of a function if it's used in an index expression and is not IMMUTABLE

On Wed, Jul 9, 2025 at 12:11 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

"David G. Johnston" <david.g.johnston@gmail.com> writes:

On Wed, Jul 9, 2025 at 9:15 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

TBH, I find this proposal to be useless nannyism.

Isn't preventing a dump-restore hazard sufficient reason to do this?

No, I don't think so. If you're not being very careful about revising
functions used in indexes, you are going to have problems a lot sooner
than some future dump/restore cycle.

Then probably this patch can just update the create index documentation to
say "create or replace it" instead of just "create it". The fact that
during 'update' (replace) existing non-implied settings can be replaced
with implied ones is a beginner/inattentive foot-gun. We do make the point
clearly in create function but it seems worthwhile to reinforce it here too.

"To use a user-defined function in an index expression or WHERE clause,
remember to mark the function immutable when you create [or replace] it."

David J.