Review: Fix snapshot taking inconsistencies

Started by Steve Singerover 15 years ago47 messages
#1Steve Singer
ssinger_pg@sympatico.ca

This is my review on the "Fix snapshot taking inconsistencies patch".

The patch applies against master (a13f12b3a18da0a61571cb134fdecea03a10d6f)

However initdb fails with:

FATAL: return type mismatch in function declared to return record
DETAIL: Function's final statement must be SELECT or INSERT/UPDATE/DELETE
RETURNING.
CONTEXT: SQL function "ts_debug"

If I run initdb without the patch applied then apply the patch I can test
the patch. However the regression tests won't run with the patch applied
because they call initdb.

Submission:
============
The patch does not include any documentation updates. I don't feel they
are required. I can't find anywhere in the documentation where it says
to expect the current behavior.

The patch does not include any regression tests. I don't think we have other
tests that (can?) test concurrency patterns like this.

Usability review:
=================
The original thread on hackers debated between changing the behavior of
explain vs changing how rules get executed (so they get a new snapshot). The
consensus seemed to be to leave explain analyze alone and change the current
behavior for rules. This is the route this patch took.

Are there any dangers: Per Marko's comments on the most recent patch:
"This patch still silently breaks pg_parse_and_rewrite()..." this still
seems unresolved. Marko proposed replacing this with something new for SQL
functions. Unfortunately I don't see this as having been followed up on.

I also don't have enough understanding of the code to see exactly how/why it
was broken or what would be involved in fixing it. I don't know if this is
why initdb is failing.

With the patch applied SQL functions still can see changes made by other
transactions after the function starts (ie a function select
$$ pg_sleep(5);insert into bar select * FROM baz;$$ inserts the row into
bar both with and without the patch applied. This is how I would expect a
function (in SQL or any pl) to work.

Does this patch effect anything outside of rules execution? Not that I've
been able to find but I'm probably not qualified to answer that and the
regression tests don't work.

Performance Review
------------------
I don't see this patch impacting performance

Coding Review
------------------
Coding guidelines: no issues that I can find
Portability: no issues
Windows: not tested but I don't see anything that looks suspicious
Comments: no obvious places that require more comments. I don't feel I have
a good enough handle on the code to judge the accuracy of the comments.
Does it do what it says: yes
compiler warnings: no
can I make it crash: initdb issue.

#2Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Steve Singer (#1)
Re: Review: Fix snapshot taking inconsistencies

On 2010-10-03 5:08 AM +0300, Steve Singer wrote:

The patch applies against master (a13f12b3a18da0a61571cb134fdecea03a10d6f)

However initdb fails with:

FATAL: return type mismatch in function declared to return record
DETAIL: Function's final statement must be SELECT or INSERT/UPDATE/DELETE
RETURNING.
CONTEXT: SQL function "ts_debug"

If I run initdb without the patch applied then apply the patch I can test
the patch. However the regression tests won't run with the patch applied
because they call initdb.

Hmm.. I can't reproduce this. What platform are you on?

The patch does not include any regression tests. I don't think we have other
tests that (can?) test concurrency patterns like this.

Right, we can't, at least not yet.

Are there any dangers: Per Marko's comments on the most recent patch:
"This patch still silently breaks pg_parse_and_rewrite()..." this still
seems unresolved. Marko proposed replacing this with something new for SQL
functions. Unfortunately I don't see this as having been followed up on.

I also don't have enough understanding of the code to see exactly how/why it
was broken or what would be involved in fixing it.

Currently pg_parse_and_rewrite() returns all Query nodes in one huge
list. That's not acceptable for this patch since that list is already
missing the information we need: when should we take a new snapshot? So
the patch breaks the API of pg_parse_and_rewrite() to return a list of
lists instead, but I'm not convinced that's a bright idea since third
party code might use it, so I suggested adding a new function. Then
again, third party code can't use pg_parse_and_rewrite() any way if/when
the wCTE patch goes in.

Regards,
Marko Tiikkaja

#3Steve Singer
ssinger_pg@sympatico.ca
In reply to: Marko Tiikkaja (#2)
Re: Review: Fix snapshot taking inconsistencies

On Mon, 4 Oct 2010, Marko Tiikkaja wrote:

On 2010-10-03 5:08 AM +0300, Steve Singer wrote:

Hmm.. I can't reproduce this. What platform are you on?

Sorry, I it seems the changes to one file (pg_proc.c) didn't get applied to
my source repository. Now that I've applied them initdb works and the
regression tests pass.

I also noticed that functions.c is now generating a warning that should be
easy to clean up.

functions.c: In function 'sql_exec_error_callback':
functions.c:989: warning: 'es' may be used uninitialized in this function
functions.c: In function 'fmgr_sql':
functions.c:712: warning: 'es' is used uninitialized in this function

Currently pg_parse_and_rewrite() returns all Query nodes in one huge list.
That's not acceptable for this patch since that list is already missing the
information we need: when should we take a new snapshot? So the patch breaks
the API of pg_parse_and_rewrite() to return a list of lists instead, but I'm
not convinced that's a bright idea since third party code might use it, so I
suggested adding a new function. Then again, third party code can't use
pg_parse_and_rewrite() any way if/when the wCTE patch goes in.

Is there any third party code in particular that your thinking of? I don't
see anything that says pg_parse_and_rewrite is part of a stable server
side API (in contrast to SPI or something an third party index access method
or custom data-type would call).

Regards,
Marko Tiikkaja

Steve Singer

#4Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Steve Singer (#3)
1 attachment(s)
Re: Review: Fix snapshot taking inconsistencies

On 2010-10-04 6:19 AM, Steve Singer wrote:

I also noticed that functions.c is now generating a warning that should be
easy to clean up.

functions.c: In function 'sql_exec_error_callback':
functions.c:989: warning: 'es' may be used uninitialized in this function
functions.c: In function 'fmgr_sql':
functions.c:712: warning: 'es' is used uninitialized in this function

Those didn't look like actual bugs to me, but fixed in the attached
patch. Thanks.

Currently pg_parse_and_rewrite() returns all Query nodes in one huge list.
That's not acceptable for this patch since that list is already missing the
information we need: when should we take a new snapshot? So the patch breaks
the API of pg_parse_and_rewrite() to return a list of lists instead, but I'm
not convinced that's a bright idea since third party code might use it, so I
suggested adding a new function. Then again, third party code can't use
pg_parse_and_rewrite() any way if/when the wCTE patch goes in.

Is there any third party code in particular that your thinking of? I don't
see anything that says pg_parse_and_rewrite is part of a stable server
side API (in contrast to SPI or something an third party index access method
or custom data-type would call).

Nope. I think I grepped contrib/ at some point and none of those were
using pg_parse_and_rewrite() so this is all just speculation. And yes,
it's not really part of any stable API but breaking third party modules
needlessly is not something I want to do. However, in this case there
is no way to avoid breaking them.

My primary concern is that any module using pg_parse_and_rewrite() will
more or less silently break once this patch goes in no matter what we
do. If we leave pg_parse_and_rewrite as is, they will do the wrong
thing (grab a new snapshot for every rewrite product). The problem
might not be noticeable (no reports of EXPLAIN ANALYZE behaving
differently for several years), but once the wCTE patch gets in, it will
be. If we modify pg_parse_and_rewrite like the patch does, modules
start behaving weirdly. So what I'm suggesting is:

- Deprecate pg_parse_and_rewrite(). I have no idea how the project
has done this in the past, but grepping the source code for
"deprecated" suggests that we just remove the function.

- Introduce a new function, specifically designed for SQL functions.
Both callers of pg_parse_and_rewrite (init_sql_fcache and
fmgr_sql_validator) call check_sql_fn_retval after
pg_parse_and_rewrite so I think we could merge those into the new
function.

Does anyone have any concerns about this? Better ideas?

Regards,
Marko Tiikkaja

Attachments:

snapshot2.patchtext/plain; charset=iso-8859-1; name=snapshot2.patchDownload
*** a/src/backend/catalog/pg_proc.c
--- b/src/backend/catalog/pg_proc.c
***************
*** 832,838 **** fmgr_sql_validator(PG_FUNCTION_ARGS)
  												  proc->proargtypes.values,
  												  proc->pronargs);
  			(void) check_sql_fn_retval(funcoid, proc->prorettype,
! 									   querytree_list,
  									   NULL, NULL);
  		}
  		else
--- 832,838 ----
  												  proc->proargtypes.values,
  												  proc->pronargs);
  			(void) check_sql_fn_retval(funcoid, proc->prorettype,
! 									   llast(querytree_list),
  									   NULL, NULL);
  		}
  		else
*** a/src/backend/executor/functions.c
--- b/src/backend/executor/functions.c
***************
*** 90,107 **** typedef struct
  	ParamListInfo paramLI;		/* Param list representing current args */
  
  	Tuplestorestate *tstore;	/* where we accumulate result tuples */
  
  	JunkFilter *junkFilter;		/* will be NULL if function returns VOID */
  
! 	/* head of linked list of execution_state records */
! 	execution_state *func_state;
  } SQLFunctionCache;
  
  typedef SQLFunctionCache *SQLFunctionCachePtr;
  
  
  /* non-export function prototypes */
! static execution_state *init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK);
  static void init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK);
--- 90,107 ----
  	ParamListInfo paramLI;		/* Param list representing current args */
  
  	Tuplestorestate *tstore;	/* where we accumulate result tuples */
+ 	Snapshot	snapshot;
  
  	JunkFilter *junkFilter;		/* will be NULL if function returns VOID */
  
! 	List *func_state;
  } SQLFunctionCache;
  
  typedef SQLFunctionCache *SQLFunctionCachePtr;
  
  
  /* non-export function prototypes */
! static List *init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK);
  static void init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK);
***************
*** 123,183 **** static void sqlfunction_destroy(DestReceiver *self);
  
  
  /* Set up the list of per-query execution_state records for a SQL function */
! static execution_state *
  init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK)
  {
! 	execution_state *firstes = NULL;
! 	execution_state *preves = NULL;
  	execution_state *lasttages = NULL;
! 	ListCell   *qtl_item;
  
! 	foreach(qtl_item, queryTree_list)
  	{
! 		Query	   *queryTree = (Query *) lfirst(qtl_item);
! 		Node	   *stmt;
! 		execution_state *newes;
  
! 		Assert(IsA(queryTree, Query));
  
! 		if (queryTree->commandType == CMD_UTILITY)
! 			stmt = queryTree->utilityStmt;
! 		else
! 			stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
  
! 		/* Precheck all commands for validity in a function */
! 		if (IsA(stmt, TransactionStmt))
! 			ereport(ERROR,
! 					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 			/* translator: %s is a SQL statement name */
! 					 errmsg("%s is not allowed in a SQL function",
! 							CreateCommandTag(stmt))));
  
! 		if (fcache->readonly_func && !CommandIsReadOnly(stmt))
! 			ereport(ERROR,
! 					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 			/* translator: %s is a SQL statement name */
! 					 errmsg("%s is not allowed in a non-volatile function",
! 							CreateCommandTag(stmt))));
  
! 		newes = (execution_state *) palloc(sizeof(execution_state));
! 		if (preves)
! 			preves->next = newes;
! 		else
! 			firstes = newes;
  
! 		newes->next = NULL;
! 		newes->status = F_EXEC_START;
! 		newes->setsResult = false;		/* might change below */
! 		newes->lazyEval = false;	/* might change below */
! 		newes->stmt = stmt;
! 		newes->qd = NULL;
  
! 		if (queryTree->canSetTag)
! 			lasttages = newes;
  
! 		preves = newes;
  	}
  
  	/*
--- 123,200 ----
  
  
  /* Set up the list of per-query execution_state records for a SQL function */
! static List *
  init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK)
  {
! 	execution_state *firstes;
! 	execution_state *preves;
  	execution_state *lasttages = NULL;
! 	List	   *eslist;
! 	ListCell   *lc1;
! 	ListCell   *lc2;
! 	List	   *qtlist;
! 	Query	   *queryTree;
! 
! 
! 	eslist = NIL;
  
! 	foreach(lc1, queryTree_list)
  	{
! 		qtlist = (List *) lfirst(lc1);
! 		firstes = NULL;
! 		preves = NULL;
  
! 		foreach(lc2, qtlist)
! 		{
! 			Node	   *stmt;
! 			execution_state *newes;
  
! 			queryTree = (Query *) lfirst(lc2);
  
! 			Assert(IsA(queryTree, Query));
  
! 			if (queryTree->commandType == CMD_UTILITY)
! 				stmt = queryTree->utilityStmt;
! 			else
! 				stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
  
! 			/* Precheck all commands for validity in a function */
! 			if (IsA(stmt, TransactionStmt))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 				/* translator: %s is a SQL statement name */
! 						 errmsg("%s is not allowed in a SQL function",
! 								CreateCommandTag(stmt))));
  
! 			if (fcache->readonly_func && !CommandIsReadOnly(stmt))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 				/* translator: %s is a SQL statement name */
! 						 errmsg("%s is not allowed in a non-volatile function",
! 								CreateCommandTag(stmt))));
! 
! 			newes = (execution_state *) palloc(sizeof(execution_state));
! 			if (preves)
! 				preves->next = newes;
! 			else
! 				firstes = newes;
  
! 			newes->next = NULL;
! 			newes->status = F_EXEC_START;
! 			newes->setsResult = false;		/* might change below */
! 			newes->lazyEval = false;	/* might change below */
! 			newes->stmt = stmt;
! 			newes->qd = NULL;
  
! 			if (queryTree->canSetTag)
! 				lasttages = newes;
! 
! 			preves = newes;
! 		}
! 
! 		eslist = lappend(eslist, firstes);
  	}
  
  	/*
***************
*** 210,216 **** init_execution_state(List *queryTree_list,
  		}
  	}
  
! 	return firstes;
  }
  
  /* Initialize the SQLFunctionCache for a SQL function */
--- 227,233 ----
  		}
  	}
  
! 	return eslist;
  }
  
  /* Initialize the SQLFunctionCache for a SQL function */
***************
*** 342,348 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  	 */
  	fcache->returnsTuple = check_sql_fn_retval(foid,
  											   rettype,
! 											   queryTree_list,
  											   NULL,
  											   &fcache->junkFilter);
  
--- 359,365 ----
  	 */
  	fcache->returnsTuple = check_sql_fn_retval(foid,
  											   rettype,
! 											   llast(queryTree_list),
  											   NULL,
  											   &fcache->junkFilter);
  
***************
*** 374,397 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  static void
  postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  {
- 	Snapshot	snapshot;
  	DestReceiver *dest;
  
  	Assert(es->qd == NULL);
  
! 	/*
! 	 * In a read-only function, use the surrounding query's snapshot;
! 	 * otherwise take a new snapshot for each query.  The snapshot should
! 	 * include a fresh command ID so that all work to date in this transaction
! 	 * is visible.
! 	 */
! 	if (fcache->readonly_func)
! 		snapshot = GetActiveSnapshot();
! 	else
! 	{
! 		CommandCounterIncrement();
! 		snapshot = GetTransactionSnapshot();
! 	}
  
  	/*
  	 * If this query produces the function result, send its output to the
--- 391,401 ----
  static void
  postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  {
  	DestReceiver *dest;
  
  	Assert(es->qd == NULL);
  
! 	Assert(ActiveSnapshotSet());
  
  	/*
  	 * If this query produces the function result, send its output to the
***************
*** 415,427 **** postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  	if (IsA(es->stmt, PlannedStmt))
  		es->qd = CreateQueryDesc((PlannedStmt *) es->stmt,
  								 fcache->src,
! 								 snapshot, InvalidSnapshot,
  								 dest,
  								 fcache->paramLI, 0);
  	else
  		es->qd = CreateUtilityQueryDesc(es->stmt,
  										fcache->src,
! 										snapshot,
  										dest,
  										fcache->paramLI);
  
--- 419,432 ----
  	if (IsA(es->stmt, PlannedStmt))
  		es->qd = CreateQueryDesc((PlannedStmt *) es->stmt,
  								 fcache->src,
! 								 GetActiveSnapshot(),
! 								 InvalidSnapshot,
  								 dest,
  								 fcache->paramLI, 0);
  	else
  		es->qd = CreateUtilityQueryDesc(es->stmt,
  										fcache->src,
! 										GetActiveSnapshot(),
  										dest,
  										fcache->paramLI);
  
***************
*** 617,622 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 622,629 ----
  	execution_state *es;
  	TupleTableSlot *slot;
  	Datum		result;
+ 	List		*eslist;
+ 	ListCell    *eslc;
  
  	/*
  	 * Switch to context in which the fcache lives.  This ensures that
***************
*** 668,680 **** fmgr_sql(PG_FUNCTION_ARGS)
  		init_sql_fcache(fcinfo->flinfo, lazyEvalOK);
  		fcache = (SQLFunctionCachePtr) fcinfo->flinfo->fn_extra;
  	}
! 	es = fcache->func_state;
  
  	/*
  	 * Convert params to appropriate format if starting a fresh execution. (If
  	 * continuing execution, we can re-use prior params.)
  	 */
! 	if (es && es->status == F_EXEC_START)
  		postquel_sub_params(fcache, fcinfo);
  
  	/*
--- 675,687 ----
  		init_sql_fcache(fcinfo->flinfo, lazyEvalOK);
  		fcache = (SQLFunctionCachePtr) fcinfo->flinfo->fn_extra;
  	}
! 	eslist = fcache->func_state;
  
  	/*
  	 * Convert params to appropriate format if starting a fresh execution. (If
  	 * continuing execution, we can re-use prior params.)
  	 */
! 	if (linitial(eslist) && ((execution_state *) linitial(eslist))->status == F_EXEC_START)
  		postquel_sub_params(fcache, fcinfo);
  
  	/*
***************
*** 687,694 **** fmgr_sql(PG_FUNCTION_ARGS)
  	/*
  	 * Find first unfinished query in function.
  	 */
! 	while (es && es->status == F_EXEC_DONE)
! 		es = es->next;
  
  	/*
  	 * Execute each command in the function one after another until we either
--- 694,711 ----
  	/*
  	 * Find first unfinished query in function.
  	 */
! 
! 	es = NULL; /* keep compiler quiet */
! 	foreach(eslc, eslist)
! 	{
! 		es = (execution_state *) lfirst(eslc);
! 
! 		while (es && es->status == F_EXEC_DONE)
! 			es = es->next;
! 
! 		if (es)
! 			break;
! 	}
  
  	/*
  	 * Execute each command in the function one after another until we either
***************
*** 699,706 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 716,746 ----
  		bool		completed;
  
  		if (es->status == F_EXEC_START)
+ 		{
+ 			if (!fcache->readonly_func)
+ 			{
+ 				/*
+ 				 * In a read-only function, use the surrounding query's snapshot;
+ 				 * otherwise take a new snapshot if we don't have one yet.  The
+ 				 * snapshot should include a fresh command ID so that all work to
+ 				 * date in this transaction is visible.
+ 				 */
+ 				if (!fcache->snapshot)
+ 				{
+ 					CommandCounterIncrement();
+ 					fcache->snapshot = RegisterSnapshot(GetTransactionSnapshot());
+ 					PushActiveSnapshot(fcache->snapshot);
+ 				}
+ 				else
+ 					PushUpdatedSnapshot(fcache->snapshot);
+ 			}
+ 			
  			postquel_start(es, fcache);
  
+ 			if (!fcache->readonly_func)
+ 				PopActiveSnapshot();
+ 		}
+ 
  		completed = postquel_getnext(es, fcache);
  
  		/*
***************
*** 726,731 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 766,790 ----
  		if (es->status != F_EXEC_DONE)
  			break;
  		es = es->next;
+ 
+ 		if (!es)
+ 		{
+ 			eslc = lnext(eslc);
+ 			if (!eslc)
+ 				break;
+ 
+ 			es = (execution_state *) lfirst(eslc);
+ 
+ 			/* make sure we take a new snapshot for this query list */
+ 			if (!fcache->readonly_func)
+ 			{
+ 				Assert(fcache->snapshot != InvalidSnapshot);
+ 				UnregisterSnapshot(fcache->snapshot);
+ 				fcache->snapshot = InvalidSnapshot;
+ 			}
+ 			else
+ 				Assert(fcache->snapshot == InvalidSnapshot);
+ 		}
  	}
  
  	/*
***************
*** 794,799 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 853,863 ----
  											  PointerGetDatum(fcache));
  				fcache->shutdown_reg = false;
  			}
+ 
+ 			/* Unregister snapshot if we have one */
+ 			if (fcache->snapshot != InvalidSnapshot)
+ 				UnregisterSnapshot(fcache->snapshot);
+ 			fcache->snapshot = InvalidSnapshot;
  		}
  		else
  		{
***************
*** 820,825 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 884,894 ----
  											  PointerGetDatum(fcache));
  				fcache->shutdown_reg = false;
  			}
+ 
+ 			/* Unregister snapshot if we have one */
+ 			if (fcache->snapshot != InvalidSnapshot)
+ 				UnregisterSnapshot(fcache->snapshot);
+ 			fcache->snapshot = InvalidSnapshot;
  		}
  	}
  	else
***************
*** 850,855 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 919,929 ----
  
  		/* Clear the tuplestore, but keep it for next time */
  		tuplestore_clear(fcache->tstore);
+ 
+ 		/* Unregister snapshot if we have one */
+ 		if (fcache->snapshot != InvalidSnapshot)
+ 			UnregisterSnapshot(fcache->snapshot);
+ 		fcache->snapshot = InvalidSnapshot;
  	}
  
  	/*
***************
*** 858,868 **** fmgr_sql(PG_FUNCTION_ARGS)
  	 */
  	if (es == NULL)
  	{
! 		es = fcache->func_state;
! 		while (es)
  		{
! 			es->status = F_EXEC_START;
! 			es = es->next;
  		}
  	}
  
--- 932,945 ----
  	 */
  	if (es == NULL)
  	{
! 		foreach(eslc, fcache->func_state)
  		{
! 			es = (execution_state *) lfirst(eslc);
! 			while (es)
! 			{
! 				es->status = F_EXEC_START;
! 				es = es->next;
! 			}
  		}
  	}
  
***************
*** 913,931 **** sql_exec_error_callback(void *arg)
  	{
  		execution_state *es;
  		int			query_num;
  
! 		es = fcache->func_state;
  		query_num = 1;
! 		while (es)
  		{
! 			if (es->qd)
  			{
! 				errcontext("SQL function \"%s\" statement %d",
! 						   fcache->fname, query_num);
! 				break;
  			}
- 			es = es->next;
- 			query_num++;
  		}
  		if (es == NULL)
  		{
--- 990,1013 ----
  	{
  		execution_state *es;
  		int			query_num;
+ 		ListCell   *lc;
  
! 		es = NULL; /* keep compiler quiet */
  		query_num = 1;
! 		foreach(lc, fcache->func_state)
  		{
! 			es = (execution_state *) lfirst(lc);
! 			while (es)
  			{
! 				if (es->qd)
! 				{
! 					errcontext("SQL function \"%s\" statement %d",
! 							   fcache->fname, query_num);
! 					break;
! 				}
! 				es = es->next;
! 				query_num++;
  			}
  		}
  		if (es == NULL)
  		{
***************
*** 956,973 **** static void
  ShutdownSQLFunction(Datum arg)
  {
  	SQLFunctionCachePtr fcache = (SQLFunctionCachePtr) DatumGetPointer(arg);
! 	execution_state *es = fcache->func_state;
  
! 	while (es != NULL)
  	{
! 		/* Shut down anything still running */
! 		if (es->status == F_EXEC_RUN)
! 			postquel_end(es);
! 		/* Reset states to START in case we're called again */
! 		es->status = F_EXEC_START;
! 		es = es->next;
  	}
  
  	/* Release tuplestore if we have one */
  	if (fcache->tstore)
  		tuplestore_end(fcache->tstore);
--- 1038,1066 ----
  ShutdownSQLFunction(Datum arg)
  {
  	SQLFunctionCachePtr fcache = (SQLFunctionCachePtr) DatumGetPointer(arg);
! 	execution_state *es;
! 	ListCell		*lc;
  
! 	foreach(lc, fcache->func_state)
  	{
! 		es = (execution_state *) lfirst(lc);
! 
! 		while (es)
! 		{
! 			/* Shut down anything still running */
! 			if (es->status == F_EXEC_RUN)
! 				postquel_end(es);
! 			/* Reset states to START in case we're called again */
! 			es->status = F_EXEC_START;
! 			es = es->next;
! 		}
  	}
  
+ 	/* Unregister snapshot if we have one */
+ 	if (fcache->snapshot != InvalidSnapshot)
+ 		UnregisterSnapshot(fcache->snapshot);
+ 	fcache->snapshot = InvalidSnapshot;
+ 
  	/* Release tuplestore if we have one */
  	if (fcache->tstore)
  		tuplestore_end(fcache->tstore);
*** a/src/backend/executor/spi.c
--- b/src/backend/executor/spi.c
***************
*** 1769,1774 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
--- 1769,1775 ----
  	SPITupleTable *my_tuptable = NULL;
  	int			res = 0;
  	bool		have_active_snap = ActiveSnapshotSet();
+ 	bool		registered_snap = false;
  	ErrorContextCallback spierrcontext;
  	CachedPlan *cplan = NULL;
  	ListCell   *lc1;
***************
*** 1872,1879 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
  				}
  				else
  				{
! 					PushActiveSnapshot(GetTransactionSnapshot());
  					pushed_active_snap = true;
  				}
  			}
  			else
--- 1873,1882 ----
  				}
  				else
  				{
! 					snapshot = RegisterSnapshot(GetTransactionSnapshot());
! 					PushActiveSnapshot(snapshot);
  					pushed_active_snap = true;
+ 					registered_snap = true;
  				}
  			}
  			else
***************
*** 1966,1975 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
--- 1969,1991 ----
  		 */
  		if (!read_only)
  			CommandCounterIncrement();
+ 
+ 		/*
+ 		 * If we took a new snapshot for this query list, unregister it and
+ 		 * make sure we take a new one for the next list.
+ 		 */
+ 		if (registered_snap)
+ 		{
+ 			UnregisterSnapshot(snapshot);
+ 			snapshot = InvalidSnapshot;
+ 		}
  	}
  
  fail:
  
+ 	if (registered_snap)
+ 		UnregisterSnapshot(snapshot);
+ 
  	/* We no longer need the cached plan refcount, if any */
  	if (cplan)
  		ReleaseCachedPlan(cplan, true);
*** a/src/backend/tcop/postgres.c
--- b/src/backend/tcop/postgres.c
***************
*** 537,547 **** pg_parse_and_rewrite(const char *query_string,	/* string to execute */
  	{
  		Node	   *parsetree = (Node *) lfirst(list_item);
  
! 		querytree_list = list_concat(querytree_list,
! 									 pg_analyze_and_rewrite(parsetree,
! 															query_string,
! 															paramTypes,
! 															numParams));
  	}
  
  	return querytree_list;
--- 537,547 ----
  	{
  		Node	   *parsetree = (Node *) lfirst(list_item);
  
! 		querytree_list = lappend(querytree_list,
! 								 pg_analyze_and_rewrite(parsetree,
! 														query_string,
! 														paramTypes,
! 														numParams));
  	}
  
  	return querytree_list;
*** a/src/backend/tcop/pquery.c
--- b/src/backend/tcop/pquery.c
***************
*** 170,180 **** ProcessQuery(PlannedStmt *plan,
  	elog(DEBUG3, "ProcessQuery");
  
  	/*
- 	 * Must always set a snapshot for plannable queries.
- 	 */
- 	PushActiveSnapshot(GetTransactionSnapshot());
- 
- 	/*
  	 * Create the QueryDesc object
  	 */
  	queryDesc = CreateQueryDesc(plan, sourceText,
--- 170,175 ----
***************
*** 234,241 **** ProcessQuery(PlannedStmt *plan,
  	/* Now take care of any queued AFTER triggers */
  	AfterTriggerEndQuery(queryDesc->estate);
  
- 	PopActiveSnapshot();
- 
  	/*
  	 * Now, we close down all the scans and free allocated resources.
  	 */
--- 229,234 ----
***************
*** 1220,1225 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1213,1219 ----
  			   char *completionTag)
  {
  	ListCell   *stmtlist_item;
+ 	Snapshot	snapshot = InvalidSnapshot;
  
  	/*
  	 * If the destination is DestRemoteExecute, change to DestNone.  The
***************
*** 1262,1267 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1256,1270 ----
  			if (log_executor_stats)
  				ResetUsage();
  
+ 			/* if no snapshot is set, grab a new one and register it */
+ 			if (snapshot == InvalidSnapshot)
+ 			{
+ 				snapshot = RegisterSnapshot(GetTransactionSnapshot());
+ 				PushActiveSnapshot(snapshot);
+ 			}
+ 			else
+ 				PushUpdatedSnapshot(snapshot);
+ 
  			if (pstmt->canSetTag)
  			{
  				/* statement can set tag string */
***************
*** 1279,1284 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1282,1289 ----
  							 altdest, NULL);
  			}
  
+ 			PopActiveSnapshot();
+ 
  			if (log_executor_stats)
  				ShowUsage("EXECUTOR STATISTICS");
  
***************
*** 1291,1301 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1296,1320 ----
  			 *
  			 * These are assumed canSetTag if they're the only stmt in the
  			 * portal.
+ 			 *
+ 			 * NotifyStmt is the only utility statement allowed in a list of
+ 			 * rewritten queries, and it doesn't need a snapshot so we don't
+ 			 * need to worry about it.  However, if the list has only one
+ 			 * statement and it's a utility statement, we are not allowed to
+ 			 * take a snapshot.  See the first comment in PortalRunUtility().
  			 */
  			if (list_length(portal->stmts) == 1)
+ 			{
+ 				Assert(snapshot == InvalidSnapshot);
+ 
  				PortalRunUtility(portal, stmt, isTopLevel, dest, completionTag);
+ 			}
  			else
+ 			{
+ 				Assert(IsA(stmt, NotifyStmt));
+ 
  				PortalRunUtility(portal, stmt, isTopLevel, altdest, NULL);
+ 			}
  		}
  
  		/*
***************
*** 1313,1318 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1332,1340 ----
  		MemoryContextDeleteChildren(PortalGetHeapMemory(portal));
  	}
  
+ 	if (snapshot != InvalidSnapshot)
+ 		UnregisterSnapshot(snapshot);
+ 
  	/*
  	 * If a command completion tag was supplied, use it.  Otherwise use the
  	 * portal's commandTag as the default completion tag.
#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#4)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2010-10-04 6:19 AM, Steve Singer wrote:

Is there any third party code in particular that your thinking of? I don't
see anything that says pg_parse_and_rewrite is part of a stable server
side API (in contrast to SPI or something an third party index access method
or custom data-type would call).

Nope. I think I grepped contrib/ at some point and none of those were
using pg_parse_and_rewrite() so this is all just speculation. And yes,
it's not really part of any stable API but breaking third party modules
needlessly is not something I want to do. However, in this case there
is no way to avoid breaking them.

The important thing in such cases is to not break third-party code
*silently*. You want to make sure that they'll get a compilation
error if they haven't adjusted their code. Change the parameter
list or invent a new name for the function.

In the particular case at hand here, I rather wonder why SQL functions
are depending on postgres.c at all. It might be better to just
duplicate a bit of code to make them independent. pg_parse_and_rewrite
would then be dead code and could be deleted.

regards, tom lane

#6Steve Singer
ssinger_pg@sympatico.ca
In reply to: Marko Tiikkaja (#4)
Re: Review: Fix snapshot taking inconsistencies

On Mon, 4 Oct 2010, Marko Tiikkaja wrote:

the patch does, modules start behaving weirdly. So what I'm suggesting is:

- Deprecate pg_parse_and_rewrite(). I have no idea how the project
has done this in the past, but grepping the source code for
"deprecated" suggests that we just remove the function.

- Introduce a new function, specifically designed for SQL functions.
Both callers of pg_parse_and_rewrite (init_sql_fcache and
fmgr_sql_validator) call check_sql_fn_retval after
pg_parse_and_rewrite so I think we could merge those into the new
function.

Does anyone have any concerns about this? Better ideas?

The only comment I've seen on this was from Tom and his only concern was
that old code wouldn't be able to compile against a new version of the
function. What you propose (delete pg_parse_and_rewrite) and replacing it
with a function of the new name sounds fine.

Since no one else has proposed a better idea and the commit fest is ticking
away I think you should go ahead and do that.

Regards,
Marko Tiikkaja

Steve

#7Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Tom Lane (#5)
Re: Review: Fix snapshot taking inconsistencies

On 2010-10-04 5:31 PM +0300, Tom Lane wrote:

Marko Tiikkaja<marko.tiikkaja@cs.helsinki.fi> writes:

Nope. I think I grepped contrib/ at some point and none of those were
using pg_parse_and_rewrite() so this is all just speculation. And yes,
it's not really part of any stable API but breaking third party modules
needlessly is not something I want to do. However, in this case there
is no way to avoid breaking them.

In the particular case at hand here, I rather wonder why SQL functions
are depending on postgres.c at all. It might be better to just
duplicate a bit of code to make them independent. pg_parse_and_rewrite
would then be dead code and could be deleted.

I'm confused. Even if we get rid of pg_parse_and_rewrite, SQL functions
need pg_parse_query and pg_analyze_and_rewrite from postgres.c. You're
not suggesting duplicating the code in those two, are you?

Regards,
Marko Tiikkaja

#8Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Steve Singer (#6)
1 attachment(s)
Re: Review: Fix snapshot taking inconsistencies

On 2010-10-07 5:21 AM +0300, Steve Singer wrote:

Since no one else has proposed a better idea and the commit fest is ticking
away I think you should go ahead and do that.

Here's a new version of the patch, deprecating pg_parse_and_rewrite.

I duplicated the parse/rewrite logic in the two places where
pg_parse_and_rewrite is currently used, per comment from Tom.

Regards,
Marko Tiikkaja

Attachments:

snapshot3.patchtext/plain; charset=iso-8859-1; name=snapshot3.patch; x-mac-creator=0; x-mac-type=0Download
*** a/src/backend/catalog/pg_proc.c
--- b/src/backend/catalog/pg_proc.c
***************
*** 755,761 **** fmgr_sql_validator(PG_FUNCTION_ARGS)
--- 755,763 ----
  	Oid			funcoid = PG_GETARG_OID(0);
  	HeapTuple	tuple;
  	Form_pg_proc proc;
+ 	List	   *raw_parsetree_list;
  	List	   *querytree_list;
+ 	ListCell   *list_item;
  	bool		isnull;
  	Datum		tmp;
  	char	   *prosrc;
***************
*** 828,836 **** fmgr_sql_validator(PG_FUNCTION_ARGS)
  		 */
  		if (!haspolyarg)
  		{
! 			querytree_list = pg_parse_and_rewrite(prosrc,
! 												  proc->proargtypes.values,
! 												  proc->pronargs);
  			(void) check_sql_fn_retval(funcoid, proc->prorettype,
  									   querytree_list,
  									   NULL, NULL);
--- 830,854 ----
  		 */
  		if (!haspolyarg)
  		{
! 			/*
! 			 * Parse and rewrite the queries in the function text.
! 			 *
! 			 * Even though check_sql_fn_retval is only interested in the last
! 			 * query, we analyze all of them here to check for any errors.
! 			 */
! 			raw_parsetree_list = pg_parse_query(prosrc);
! 			
! 			querytree_list = NIL;
! 			foreach(list_item, raw_parsetree_list)
! 			{
! 				Node *parsetree = (Node *) lfirst(list_item);
! 
! 				querytree_list = pg_analyze_and_rewrite(parsetree, prosrc,
! 														proc->proargtypes.values, proc->pronargs);
! 			}
! 
! 			Assert(querytree_list != NIL);
! 
  			(void) check_sql_fn_retval(funcoid, proc->prorettype,
  									   querytree_list,
  									   NULL, NULL);
*** a/src/backend/executor/functions.c
--- b/src/backend/executor/functions.c
***************
*** 90,107 **** typedef struct
  	ParamListInfo paramLI;		/* Param list representing current args */
  
  	Tuplestorestate *tstore;	/* where we accumulate result tuples */
  
  	JunkFilter *junkFilter;		/* will be NULL if function returns VOID */
  
! 	/* head of linked list of execution_state records */
! 	execution_state *func_state;
  } SQLFunctionCache;
  
  typedef SQLFunctionCache *SQLFunctionCachePtr;
  
  
  /* non-export function prototypes */
! static execution_state *init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK);
  static void init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK);
--- 90,107 ----
  	ParamListInfo paramLI;		/* Param list representing current args */
  
  	Tuplestorestate *tstore;	/* where we accumulate result tuples */
+ 	Snapshot	snapshot;
  
  	JunkFilter *junkFilter;		/* will be NULL if function returns VOID */
  
! 	List *func_state;
  } SQLFunctionCache;
  
  typedef SQLFunctionCache *SQLFunctionCachePtr;
  
  
  /* non-export function prototypes */
! static List *init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK);
  static void init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK);
***************
*** 123,183 **** static void sqlfunction_destroy(DestReceiver *self);
  
  
  /* Set up the list of per-query execution_state records for a SQL function */
! static execution_state *
  init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK)
  {
! 	execution_state *firstes = NULL;
! 	execution_state *preves = NULL;
  	execution_state *lasttages = NULL;
! 	ListCell   *qtl_item;
  
! 	foreach(qtl_item, queryTree_list)
  	{
! 		Query	   *queryTree = (Query *) lfirst(qtl_item);
! 		Node	   *stmt;
! 		execution_state *newes;
! 
! 		Assert(IsA(queryTree, Query));
! 
! 		if (queryTree->commandType == CMD_UTILITY)
! 			stmt = queryTree->utilityStmt;
! 		else
! 			stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
! 
! 		/* Precheck all commands for validity in a function */
! 		if (IsA(stmt, TransactionStmt))
! 			ereport(ERROR,
! 					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 			/* translator: %s is a SQL statement name */
! 					 errmsg("%s is not allowed in a SQL function",
! 							CreateCommandTag(stmt))));
! 
! 		if (fcache->readonly_func && !CommandIsReadOnly(stmt))
! 			ereport(ERROR,
! 					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 			/* translator: %s is a SQL statement name */
! 					 errmsg("%s is not allowed in a non-volatile function",
! 							CreateCommandTag(stmt))));
! 
! 		newes = (execution_state *) palloc(sizeof(execution_state));
! 		if (preves)
! 			preves->next = newes;
! 		else
! 			firstes = newes;
! 
! 		newes->next = NULL;
! 		newes->status = F_EXEC_START;
! 		newes->setsResult = false;		/* might change below */
! 		newes->lazyEval = false;	/* might change below */
! 		newes->stmt = stmt;
! 		newes->qd = NULL;
! 
! 		if (queryTree->canSetTag)
! 			lasttages = newes;
! 
! 		preves = newes;
  	}
  
  	/*
--- 123,200 ----
  
  
  /* Set up the list of per-query execution_state records for a SQL function */
! static List *
  init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK)
  {
! 	execution_state *firstes;
! 	execution_state *preves;
  	execution_state *lasttages = NULL;
! 	List	   *eslist;
! 	ListCell   *lc1;
! 	ListCell   *lc2;
! 	List	   *qtlist;
! 	Query	   *queryTree;
  
! 
! 	eslist = NIL;
! 
! 	foreach(lc1, queryTree_list)
  	{
! 		qtlist = (List *) lfirst(lc1);
! 		firstes = NULL;
! 		preves = NULL;
! 
! 		foreach(lc2, qtlist)
! 		{
! 			Node	   *stmt;
! 			execution_state *newes;
! 
! 			queryTree = (Query *) lfirst(lc2);
! 
! 			Assert(IsA(queryTree, Query));
! 
! 			if (queryTree->commandType == CMD_UTILITY)
! 				stmt = queryTree->utilityStmt;
! 			else
! 				stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
! 
! 			/* Precheck all commands for validity in a function */
! 			if (IsA(stmt, TransactionStmt))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 				/* translator: %s is a SQL statement name */
! 						 errmsg("%s is not allowed in a SQL function",
! 								CreateCommandTag(stmt))));
! 
! 			if (fcache->readonly_func && !CommandIsReadOnly(stmt))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 				/* translator: %s is a SQL statement name */
! 						 errmsg("%s is not allowed in a non-volatile function",
! 								CreateCommandTag(stmt))));
! 
! 			newes = (execution_state *) palloc(sizeof(execution_state));
! 			if (preves)
! 				preves->next = newes;
! 			else
! 				firstes = newes;
! 
! 			newes->next = NULL;
! 			newes->status = F_EXEC_START;
! 			newes->setsResult = false;		/* might change below */
! 			newes->lazyEval = false;	/* might change below */
! 			newes->stmt = stmt;
! 			newes->qd = NULL;
! 
! 			if (queryTree->canSetTag)
! 				lasttages = newes;
! 
! 			preves = newes;
! 		}
! 
! 		eslist = lappend(eslist, firstes);
  	}
  
  	/*
***************
*** 210,216 **** init_execution_state(List *queryTree_list,
  		}
  	}
  
! 	return firstes;
  }
  
  /* Initialize the SQLFunctionCache for a SQL function */
--- 227,233 ----
  		}
  	}
  
! 	return eslist;
  }
  
  /* Initialize the SQLFunctionCache for a SQL function */
***************
*** 224,230 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
--- 241,249 ----
  	SQLFunctionCachePtr fcache;
  	Oid		   *argOidVect;
  	int			nargs;
+ 	List	   *raw_parsetree_list;
  	List	   *queryTree_list;
+ 	ListCell   *list_item;
  	Datum		tmp;
  	bool		isNull;
  
***************
*** 319,325 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  	/*
  	 * Parse and rewrite the queries in the function text.
  	 */
! 	queryTree_list = pg_parse_and_rewrite(fcache->src, argOidVect, nargs);
  
  	/*
  	 * Check that the function returns the type it claims to.  Although in
--- 338,356 ----
  	/*
  	 * Parse and rewrite the queries in the function text.
  	 */
! 	raw_parsetree_list = pg_parse_query(fcache->src);
! 
! 	queryTree_list = NIL;
! 	foreach(list_item, raw_parsetree_list)
! 	{
! 		Node *parsetree = (Node *) lfirst(list_item);
! 
! 		queryTree_list = lappend(queryTree_list,
! 								 pg_analyze_and_rewrite(parsetree,
! 									 					fcache->src,
! 														argOidVect,
! 														nargs));
! 	}
  
  	/*
  	 * Check that the function returns the type it claims to.  Although in
***************
*** 342,348 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  	 */
  	fcache->returnsTuple = check_sql_fn_retval(foid,
  											   rettype,
! 											   queryTree_list,
  											   NULL,
  											   &fcache->junkFilter);
  
--- 373,379 ----
  	 */
  	fcache->returnsTuple = check_sql_fn_retval(foid,
  											   rettype,
! 											   llast(queryTree_list),
  											   NULL,
  											   &fcache->junkFilter);
  
***************
*** 374,397 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  static void
  postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  {
- 	Snapshot	snapshot;
  	DestReceiver *dest;
  
  	Assert(es->qd == NULL);
  
! 	/*
! 	 * In a read-only function, use the surrounding query's snapshot;
! 	 * otherwise take a new snapshot for each query.  The snapshot should
! 	 * include a fresh command ID so that all work to date in this transaction
! 	 * is visible.
! 	 */
! 	if (fcache->readonly_func)
! 		snapshot = GetActiveSnapshot();
! 	else
! 	{
! 		CommandCounterIncrement();
! 		snapshot = GetTransactionSnapshot();
! 	}
  
  	/*
  	 * If this query produces the function result, send its output to the
--- 405,415 ----
  static void
  postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  {
  	DestReceiver *dest;
  
  	Assert(es->qd == NULL);
  
! 	Assert(ActiveSnapshotSet());
  
  	/*
  	 * If this query produces the function result, send its output to the
***************
*** 415,427 **** postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  	if (IsA(es->stmt, PlannedStmt))
  		es->qd = CreateQueryDesc((PlannedStmt *) es->stmt,
  								 fcache->src,
! 								 snapshot, InvalidSnapshot,
  								 dest,
  								 fcache->paramLI, 0);
  	else
  		es->qd = CreateUtilityQueryDesc(es->stmt,
  										fcache->src,
! 										snapshot,
  										dest,
  										fcache->paramLI);
  
--- 433,446 ----
  	if (IsA(es->stmt, PlannedStmt))
  		es->qd = CreateQueryDesc((PlannedStmt *) es->stmt,
  								 fcache->src,
! 								 GetActiveSnapshot(),
! 								 InvalidSnapshot,
  								 dest,
  								 fcache->paramLI, 0);
  	else
  		es->qd = CreateUtilityQueryDesc(es->stmt,
  										fcache->src,
! 										GetActiveSnapshot(),
  										dest,
  										fcache->paramLI);
  
***************
*** 617,622 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 636,643 ----
  	execution_state *es;
  	TupleTableSlot *slot;
  	Datum		result;
+ 	List		*eslist;
+ 	ListCell    *eslc;
  
  	/*
  	 * Switch to context in which the fcache lives.  This ensures that
***************
*** 668,680 **** fmgr_sql(PG_FUNCTION_ARGS)
  		init_sql_fcache(fcinfo->flinfo, lazyEvalOK);
  		fcache = (SQLFunctionCachePtr) fcinfo->flinfo->fn_extra;
  	}
! 	es = fcache->func_state;
  
  	/*
  	 * Convert params to appropriate format if starting a fresh execution. (If
  	 * continuing execution, we can re-use prior params.)
  	 */
! 	if (es && es->status == F_EXEC_START)
  		postquel_sub_params(fcache, fcinfo);
  
  	/*
--- 689,701 ----
  		init_sql_fcache(fcinfo->flinfo, lazyEvalOK);
  		fcache = (SQLFunctionCachePtr) fcinfo->flinfo->fn_extra;
  	}
! 	eslist = fcache->func_state;
  
  	/*
  	 * Convert params to appropriate format if starting a fresh execution. (If
  	 * continuing execution, we can re-use prior params.)
  	 */
! 	if (linitial(eslist) && ((execution_state *) linitial(eslist))->status == F_EXEC_START)
  		postquel_sub_params(fcache, fcinfo);
  
  	/*
***************
*** 687,694 **** fmgr_sql(PG_FUNCTION_ARGS)
  	/*
  	 * Find first unfinished query in function.
  	 */
! 	while (es && es->status == F_EXEC_DONE)
! 		es = es->next;
  
  	/*
  	 * Execute each command in the function one after another until we either
--- 708,725 ----
  	/*
  	 * Find first unfinished query in function.
  	 */
! 
! 	es = NULL; /* keep compiler quiet */
! 	foreach(eslc, eslist)
! 	{
! 		es = (execution_state *) lfirst(eslc);
! 
! 		while (es && es->status == F_EXEC_DONE)
! 			es = es->next;
! 
! 		if (es)
! 			break;
! 	}
  
  	/*
  	 * Execute each command in the function one after another until we either
***************
*** 699,706 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 730,760 ----
  		bool		completed;
  
  		if (es->status == F_EXEC_START)
+ 		{
+ 			if (!fcache->readonly_func)
+ 			{
+ 				/*
+ 				 * In a read-only function, use the surrounding query's snapshot;
+ 				 * otherwise take a new snapshot if we don't have one yet.  The
+ 				 * snapshot should include a fresh command ID so that all work to
+ 				 * date in this transaction is visible.
+ 				 */
+ 				if (!fcache->snapshot)
+ 				{
+ 					CommandCounterIncrement();
+ 					fcache->snapshot = RegisterSnapshot(GetTransactionSnapshot());
+ 					PushActiveSnapshot(fcache->snapshot);
+ 				}
+ 				else
+ 					PushUpdatedSnapshot(fcache->snapshot);
+ 			}
+ 			
  			postquel_start(es, fcache);
  
+ 			if (!fcache->readonly_func)
+ 				PopActiveSnapshot();
+ 		}
+ 
  		completed = postquel_getnext(es, fcache);
  
  		/*
***************
*** 726,731 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 780,804 ----
  		if (es->status != F_EXEC_DONE)
  			break;
  		es = es->next;
+ 
+ 		if (!es)
+ 		{
+ 			eslc = lnext(eslc);
+ 			if (!eslc)
+ 				break;
+ 
+ 			es = (execution_state *) lfirst(eslc);
+ 
+ 			/* make sure we take a new snapshot for this query list */
+ 			if (!fcache->readonly_func)
+ 			{
+ 				Assert(fcache->snapshot != InvalidSnapshot);
+ 				UnregisterSnapshot(fcache->snapshot);
+ 				fcache->snapshot = InvalidSnapshot;
+ 			}
+ 			else
+ 				Assert(fcache->snapshot == InvalidSnapshot);
+ 		}
  	}
  
  	/*
***************
*** 794,799 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 867,877 ----
  											  PointerGetDatum(fcache));
  				fcache->shutdown_reg = false;
  			}
+ 
+ 			/* Unregister snapshot if we have one */
+ 			if (fcache->snapshot != InvalidSnapshot)
+ 				UnregisterSnapshot(fcache->snapshot);
+ 			fcache->snapshot = InvalidSnapshot;
  		}
  		else
  		{
***************
*** 820,825 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 898,908 ----
  											  PointerGetDatum(fcache));
  				fcache->shutdown_reg = false;
  			}
+ 
+ 			/* Unregister snapshot if we have one */
+ 			if (fcache->snapshot != InvalidSnapshot)
+ 				UnregisterSnapshot(fcache->snapshot);
+ 			fcache->snapshot = InvalidSnapshot;
  		}
  	}
  	else
***************
*** 850,855 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 933,943 ----
  
  		/* Clear the tuplestore, but keep it for next time */
  		tuplestore_clear(fcache->tstore);
+ 
+ 		/* Unregister snapshot if we have one */
+ 		if (fcache->snapshot != InvalidSnapshot)
+ 			UnregisterSnapshot(fcache->snapshot);
+ 		fcache->snapshot = InvalidSnapshot;
  	}
  
  	/*
***************
*** 858,868 **** fmgr_sql(PG_FUNCTION_ARGS)
  	 */
  	if (es == NULL)
  	{
! 		es = fcache->func_state;
! 		while (es)
  		{
! 			es->status = F_EXEC_START;
! 			es = es->next;
  		}
  	}
  
--- 946,959 ----
  	 */
  	if (es == NULL)
  	{
! 		foreach(eslc, fcache->func_state)
  		{
! 			es = (execution_state *) lfirst(eslc);
! 			while (es)
! 			{
! 				es->status = F_EXEC_START;
! 				es = es->next;
! 			}
  		}
  	}
  
***************
*** 913,931 **** sql_exec_error_callback(void *arg)
  	{
  		execution_state *es;
  		int			query_num;
  
! 		es = fcache->func_state;
  		query_num = 1;
! 		while (es)
  		{
! 			if (es->qd)
  			{
! 				errcontext("SQL function \"%s\" statement %d",
! 						   fcache->fname, query_num);
! 				break;
  			}
- 			es = es->next;
- 			query_num++;
  		}
  		if (es == NULL)
  		{
--- 1004,1027 ----
  	{
  		execution_state *es;
  		int			query_num;
+ 		ListCell   *lc;
  
! 		es = NULL; /* keep compiler quiet */
  		query_num = 1;
! 		foreach(lc, fcache->func_state)
  		{
! 			es = (execution_state *) lfirst(lc);
! 			while (es)
  			{
! 				if (es->qd)
! 				{
! 					errcontext("SQL function \"%s\" statement %d",
! 							   fcache->fname, query_num);
! 					break;
! 				}
! 				es = es->next;
! 				query_num++;
  			}
  		}
  		if (es == NULL)
  		{
***************
*** 956,973 **** static void
  ShutdownSQLFunction(Datum arg)
  {
  	SQLFunctionCachePtr fcache = (SQLFunctionCachePtr) DatumGetPointer(arg);
! 	execution_state *es = fcache->func_state;
  
! 	while (es != NULL)
  	{
! 		/* Shut down anything still running */
! 		if (es->status == F_EXEC_RUN)
! 			postquel_end(es);
! 		/* Reset states to START in case we're called again */
! 		es->status = F_EXEC_START;
! 		es = es->next;
  	}
  
  	/* Release tuplestore if we have one */
  	if (fcache->tstore)
  		tuplestore_end(fcache->tstore);
--- 1052,1080 ----
  ShutdownSQLFunction(Datum arg)
  {
  	SQLFunctionCachePtr fcache = (SQLFunctionCachePtr) DatumGetPointer(arg);
! 	execution_state *es;
! 	ListCell		*lc;
  
! 	foreach(lc, fcache->func_state)
  	{
! 		es = (execution_state *) lfirst(lc);
! 
! 		while (es)
! 		{
! 			/* Shut down anything still running */
! 			if (es->status == F_EXEC_RUN)
! 				postquel_end(es);
! 			/* Reset states to START in case we're called again */
! 			es->status = F_EXEC_START;
! 			es = es->next;
! 		}
  	}
  
+ 	/* Unregister snapshot if we have one */
+ 	if (fcache->snapshot != InvalidSnapshot)
+ 		UnregisterSnapshot(fcache->snapshot);
+ 	fcache->snapshot = InvalidSnapshot;
+ 
  	/* Release tuplestore if we have one */
  	if (fcache->tstore)
  		tuplestore_end(fcache->tstore);
*** a/src/backend/executor/spi.c
--- b/src/backend/executor/spi.c
***************
*** 1769,1774 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
--- 1769,1775 ----
  	SPITupleTable *my_tuptable = NULL;
  	int			res = 0;
  	bool		have_active_snap = ActiveSnapshotSet();
+ 	bool		registered_snap = false;
  	ErrorContextCallback spierrcontext;
  	CachedPlan *cplan = NULL;
  	ListCell   *lc1;
***************
*** 1872,1879 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
  				}
  				else
  				{
! 					PushActiveSnapshot(GetTransactionSnapshot());
  					pushed_active_snap = true;
  				}
  			}
  			else
--- 1873,1882 ----
  				}
  				else
  				{
! 					snapshot = RegisterSnapshot(GetTransactionSnapshot());
! 					PushActiveSnapshot(snapshot);
  					pushed_active_snap = true;
+ 					registered_snap = true;
  				}
  			}
  			else
***************
*** 1966,1975 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
--- 1969,1991 ----
  		 */
  		if (!read_only)
  			CommandCounterIncrement();
+ 
+ 		/*
+ 		 * If we took a new snapshot for this query list, unregister it and
+ 		 * make sure we take a new one for the next list.
+ 		 */
+ 		if (registered_snap)
+ 		{
+ 			UnregisterSnapshot(snapshot);
+ 			snapshot = InvalidSnapshot;
+ 		}
  	}
  
  fail:
  
+ 	if (registered_snap)
+ 		UnregisterSnapshot(snapshot);
+ 
  	/* We no longer need the cached plan refcount, if any */
  	if (cplan)
  		ReleaseCachedPlan(cplan, true);
*** a/src/backend/tcop/postgres.c
--- b/src/backend/tcop/postgres.c
***************
*** 505,552 **** client_read_ended(void)
  	}
  }
  
- 
- /*
-  * Parse a query string and pass it through the rewriter.
-  *
-  * A list of Query nodes is returned, since the string might contain
-  * multiple queries and/or the rewriter might expand one query to several.
-  *
-  * NOTE: this routine is no longer used for processing interactive queries,
-  * but it is still needed for parsing of SQL function bodies.
-  */
- List *
- pg_parse_and_rewrite(const char *query_string,	/* string to execute */
- 					 Oid *paramTypes,	/* parameter types */
- 					 int numParams)		/* number of parameters */
- {
- 	List	   *raw_parsetree_list;
- 	List	   *querytree_list;
- 	ListCell   *list_item;
- 
- 	/*
- 	 * (1) parse the request string into a list of raw parse trees.
- 	 */
- 	raw_parsetree_list = pg_parse_query(query_string);
- 
- 	/*
- 	 * (2) Do parse analysis and rule rewrite.
- 	 */
- 	querytree_list = NIL;
- 	foreach(list_item, raw_parsetree_list)
- 	{
- 		Node	   *parsetree = (Node *) lfirst(list_item);
- 
- 		querytree_list = list_concat(querytree_list,
- 									 pg_analyze_and_rewrite(parsetree,
- 															query_string,
- 															paramTypes,
- 															numParams));
- 	}
- 
- 	return querytree_list;
- }
- 
  /*
   * Do raw parsing (only).
   *
--- 505,510 ----
*** a/src/backend/tcop/pquery.c
--- b/src/backend/tcop/pquery.c
***************
*** 170,180 **** ProcessQuery(PlannedStmt *plan,
  	elog(DEBUG3, "ProcessQuery");
  
  	/*
- 	 * Must always set a snapshot for plannable queries.
- 	 */
- 	PushActiveSnapshot(GetTransactionSnapshot());
- 
- 	/*
  	 * Create the QueryDesc object
  	 */
  	queryDesc = CreateQueryDesc(plan, sourceText,
--- 170,175 ----
***************
*** 234,241 **** ProcessQuery(PlannedStmt *plan,
  	/* Now take care of any queued AFTER triggers */
  	AfterTriggerEndQuery(queryDesc->estate);
  
- 	PopActiveSnapshot();
- 
  	/*
  	 * Now, we close down all the scans and free allocated resources.
  	 */
--- 229,234 ----
***************
*** 1220,1225 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1213,1219 ----
  			   char *completionTag)
  {
  	ListCell   *stmtlist_item;
+ 	Snapshot	snapshot = InvalidSnapshot;
  
  	/*
  	 * If the destination is DestRemoteExecute, change to DestNone.  The
***************
*** 1262,1267 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1256,1270 ----
  			if (log_executor_stats)
  				ResetUsage();
  
+ 			/* if no snapshot is set, grab a new one and register it */
+ 			if (snapshot == InvalidSnapshot)
+ 			{
+ 				snapshot = RegisterSnapshot(GetTransactionSnapshot());
+ 				PushActiveSnapshot(snapshot);
+ 			}
+ 			else
+ 				PushUpdatedSnapshot(snapshot);
+ 
  			if (pstmt->canSetTag)
  			{
  				/* statement can set tag string */
***************
*** 1279,1284 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1282,1289 ----
  							 altdest, NULL);
  			}
  
+ 			PopActiveSnapshot();
+ 
  			if (log_executor_stats)
  				ShowUsage("EXECUTOR STATISTICS");
  
***************
*** 1291,1301 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1296,1320 ----
  			 *
  			 * These are assumed canSetTag if they're the only stmt in the
  			 * portal.
+ 			 *
+ 			 * NotifyStmt is the only utility statement allowed in a list of
+ 			 * rewritten queries, and it doesn't need a snapshot so we don't
+ 			 * need to worry about it.  However, if the list has only one
+ 			 * statement and it's a utility statement, we are not allowed to
+ 			 * take a snapshot.  See the first comment in PortalRunUtility().
  			 */
  			if (list_length(portal->stmts) == 1)
+ 			{
+ 				Assert(snapshot == InvalidSnapshot);
+ 
  				PortalRunUtility(portal, stmt, isTopLevel, dest, completionTag);
+ 			}
  			else
+ 			{
+ 				Assert(IsA(stmt, NotifyStmt));
+ 
  				PortalRunUtility(portal, stmt, isTopLevel, altdest, NULL);
+ 			}
  		}
  
  		/*
***************
*** 1313,1318 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1332,1340 ----
  		MemoryContextDeleteChildren(PortalGetHeapMemory(portal));
  	}
  
+ 	if (snapshot != InvalidSnapshot)
+ 		UnregisterSnapshot(snapshot);
+ 
  	/*
  	 * If a command completion tag was supplied, use it.  Otherwise use the
  	 * portal's commandTag as the default completion tag.
*** a/src/include/tcop/tcopprot.h
--- b/src/include/tcop/tcopprot.h
***************
*** 45,52 **** typedef enum
  
  extern int	log_statement;
  
- extern List *pg_parse_and_rewrite(const char *query_string,
- 					 Oid *paramTypes, int numParams);
  extern List *pg_parse_query(const char *query_string);
  extern List *pg_analyze_and_rewrite(Node *parsetree, const char *query_string,
  					   Oid *paramTypes, int numParams);
--- 45,50 ----
#9Steve Singer
ssinger_pg@sympatico.ca
In reply to: Marko Tiikkaja (#8)
Re: Review: Fix snapshot taking inconsistencies

On Sun, 10 Oct 2010, Marko Tiikkaja wrote:

On 2010-10-07 5:21 AM +0300, Steve Singer wrote:

Since no one else has proposed a better idea and the commit fest is ticking
away I think you should go ahead and do that.

Here's a new version of the patch, deprecating pg_parse_and_rewrite.

I duplicated the parse/rewrite logic in the two places where
pg_parse_and_rewrite is currently used, per comment from Tom.

That looks fine.

I've marking the patch as ready for a committer.

Show quoted text

Regards,
Marko Tiikkaja

#10Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#8)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

Here's a new version of the patch, deprecating pg_parse_and_rewrite.

I started looking at this patch, and I'm wondering why you inserted all
the Register/UnregisterSnapshot calls that weren't there before (eg, why
did spi.c have to change at all?). ISTM either these are not necessary,
or there is a pre-existing snapshot management bug that's independent
of the question of just when to take new snapshots. I couldn't find
anything in the thread claiming the latter though.

regards, tom lane

#11Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Tom Lane (#10)
Re: Review: Fix snapshot taking inconsistencies

On 2010-10-13 1:21 AM +0300, Tom Lane wrote:

Marko Tiikkaja<marko.tiikkaja@cs.helsinki.fi> writes:

Here's a new version of the patch, deprecating pg_parse_and_rewrite.

I started looking at this patch, and I'm wondering why you inserted all
the Register/UnregisterSnapshot calls that weren't there before (eg, why
did spi.c have to change at all?). ISTM either these are not necessary,
or there is a pre-existing snapshot management bug that's independent
of the question of just when to take new snapshots. I couldn't find
anything in the thread claiming the latter though.

That's actually just my ignorance I forgot to mention. As I understand
it, our code currently first pushes one snapshot and then does multiple
PushActiveSnapshot (or PushUpdatedSnapshot)/PopActiveSnapshot rounds
before popping the oldest snapshot off the stack (and releasing it). So
in the patch, I would've had to push the snapshot twice the first time
to avoid it being released.

Thinking about it now, that might be a better option. Or maybe we
should change the snapshot API to make this more convenient?

The spi.c change also changes the logic; the SPI code currently takes a
new snapshot for every query if the caller doesn't provide a snapshot.

Regards,
Marko Tiikkaja

#12Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#11)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2010-10-13 1:21 AM +0300, Tom Lane wrote:

I started looking at this patch, and I'm wondering why you inserted all
the Register/UnregisterSnapshot calls that weren't there before

That's actually just my ignorance I forgot to mention. As I understand
it, our code currently first pushes one snapshot and then does multiple
PushActiveSnapshot (or PushUpdatedSnapshot)/PopActiveSnapshot rounds
before popping the oldest snapshot off the stack (and releasing it). So
in the patch, I would've had to push the snapshot twice the first time
to avoid it being released.

It looks to me like you've added quite a lot of management overhead that
wasn't there before. Wouldn't it be better to just not pop the snapshot
till you're done with it?

Thinking about it now, that might be a better option. Or maybe we
should change the snapshot API to make this more convenient?

Well, I'm not in love with the current snapshot API by any means,
particularly not PushUpdatedSnapshot which seems to be the only
API-sanctioned way to put a new CID into a snapshot without taking
a whole new snapshot. It'd be better if the logic was something
along the lines of:

* at start of a query, PushActiveSnapshot(GetTransactionSnapshot()).

* between commands of a query, CommandCounterIncrement and
then directly modify the curcid of the active snapshot;
AFAICS there's no reason to make another copy of it at this
point. Especially not if we can see it has refcount 1.

* at end of query, PopActiveSnapshot().

where a "query" is whatever we think the unit of noticing commits by
other backends ought to be.

The spi.c change also changes the logic; the SPI code currently takes a
new snapshot for every query if the caller doesn't provide a snapshot.

[ squint... ] Oh. I see now, but that is horribly ugly and
underdocumented. The code was previously treating the snapshot argument
as a constant and relying on that constant value to tell it what to do
each time through the loop. Now you've got it changing the flag and
then changing it back sometime later. Ick.

I think what you need to do to make this understandable is to move the
snapshot push/pop logic outside the per-command loop, instead of hacking
things around to keep it exactly where it was before. We may well need
to adjust the API of snapmgr.c to make that sane.

BTW, this patch seems to be also the time to remove the AtStart_Cache()
call in CommandCounterIncrement, as foreseen in the comment there.

regards, tom lane

#13Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Tom Lane (#12)
Re: Review: Fix snapshot taking inconsistencies

On 2010-10-13 2:10 AM +0300, Tom Lane wrote:

Marko Tiikkaja<marko.tiikkaja@cs.helsinki.fi> writes:

That's actually just my ignorance I forgot to mention. As I understand
it, our code currently first pushes one snapshot and then does multiple
PushActiveSnapshot (or PushUpdatedSnapshot)/PopActiveSnapshot rounds
before popping the oldest snapshot off the stack (and releasing it). So
in the patch, I would've had to push the snapshot twice the first time
to avoid it being released.

It looks to me like you've added quite a lot of management overhead that
wasn't there before. Wouldn't it be better to just not pop the snapshot
till you're done with it?

Yes, you're absolutely right.

It'd be better if the logic was something
along the lines of:

That's exactly what I had in mind, so +1.

The spi.c change also changes the logic; the SPI code currently takes a
new snapshot for every query if the caller doesn't provide a snapshot.

[ squint... ] Oh. I see now, but that is horribly ugly and
underdocumented. The code was previously treating the snapshot argument
as a constant and relying on that constant value to tell it what to do
each time through the loop. Now you've got it changing the flag and
then changing it back sometime later. Ick.

I think what you need to do to make this understandable is to move the
snapshot push/pop logic outside the per-command loop, instead of hacking
things around to keep it exactly where it was before. We may well need
to adjust the API of snapmgr.c to make that sane.

*blushes*

Yeah, that makes a lot more sense.

BTW, this patch seems to be also the time to remove the AtStart_Cache()
call in CommandCounterIncrement, as foreseen in the comment there.

Frankly, I have no idea what to do about this.

Regards,
Marko Tiikkaja

#14Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#13)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2010-10-13 2:10 AM +0300, Tom Lane wrote:

BTW, this patch seems to be also the time to remove the AtStart_Cache()
call in CommandCounterIncrement, as foreseen in the comment there.

Frankly, I have no idea what to do about this.

Just delete the call. The only reason I didn't remove it in 2007 is
I was afraid to risk changing things in late beta; but that's not the
situation now.

regards, tom lane

#15Alvaro Herrera
alvherre@commandprompt.com
In reply to: Tom Lane (#14)
Re: Review: Fix snapshot taking inconsistencies

Excerpts from Tom Lane's message of mar oct 12 20:49:28 -0300 2010:

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2010-10-13 2:10 AM +0300, Tom Lane wrote:

BTW, this patch seems to be also the time to remove the AtStart_Cache()
call in CommandCounterIncrement, as foreseen in the comment there.

Frankly, I have no idea what to do about this.

Just delete the call. The only reason I didn't remove it in 2007 is
I was afraid to risk changing things in late beta; but that's not the
situation now.

I just applied just this change and ran the regression tests; it works
fine. I didn't do anything else though, like the cache-clobber-always
flag, etc. If no one objects I will push this patch to see what the
buildfarm has to say about it.

diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index b02db9e..d2e2e11 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -729,17 +729,6 @@ CommandCounterIncrement(void)
         */
        AtCCI_LocalCache();
    }
-
-   /*
-    * Make any other backends' catalog changes visible to me.
-    *
-    * XXX this is probably in the wrong place: CommandCounterIncrement should
-    * be purely a local operation, most likely.  However fooling with this
-    * will affect asynchronous cross-backend interactions, which doesn't seem
-    * like a wise thing to do in late beta, so save improving this for
-    * another day - tgl 2007-11-30
-    */
-   AtStart_Cache();
 }

/*

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

#16Alvaro Herrera
alvherre@commandprompt.com
In reply to: Tom Lane (#5)
2 attachment(s)
Re: Review: Fix snapshot taking inconsistencies

Excerpts from Tom Lane's message of lun oct 04 10:31:26 -0400 2010:

In the particular case at hand here, I rather wonder why SQL functions
are depending on postgres.c at all. It might be better to just
duplicate a bit of code to make them independent. pg_parse_and_rewrite
would then be dead code and could be deleted.

This idea doesn't work, unless pushed a lot further. Attached are two
separate patches, extracted from the last patch version posted by Marko
(git commit --interactive helped here). The first one does what you
suggest above: remove pg_parse_and_rewrite and inline it into the
callers. The other patch is the remainder.

Read on for the details of the first patch. As for the second patch,
which is Marko's original intention anyway, I don't see that it needs to
be delayed because of the first one. So while I haven't reviewed it, I
think it should be considered separately.

The problem with this patch (0001) is that the inlined versions of the
code that used to be pg_parse_and_rewrite are still depending on
functions in postgres.c. These are pg_parse_query and
pg_analyze_and_rewrite. pg_parse_query is just a layer on top of
raw_parser. pg_analyze_and_rewrite is a layer on top of parse_analyze
plus pg_rewrite_query (also on postgres.c).

Now, the only difference between those functions and the ones that
underlie them is that they have the bunch of tracing macros and
log_parser_stats reporting. So one solution would be to have SQL
functions (pg_proc.c and executor/functions.c) call the raw parser.c and
analyze.c functions directly, without invoking the tracing/logging code.

The other idea would be to move the whole of those functions out of
postgres.c and into their own modules, i.e. move pg_parse_query into
parser.c and pg_analyze_and_rewrite and pg_rewrite_query into
rewriteHandler.c. (This actually requires a bit more effort because we
should also move pg_analyze_and_rewrite_params out of postgres.c,
following pg_analyze_and_rewrite).

Note that pg_analyze_and_rewrite and its "params" siblings are being
called from copy.c, spi.c, optimizer/util/clauses.c, and plancache.c.
So it does make certain sense to move them out of postgres.c, if we want
to think of postgres.c as a module only concerned with client
interaction.

The only quarrel I have with this code shuffling is that
pg_rewrite_query is being called from exec_parse_message. Since it's
now a static function, it would have to stop being static so that it can
be called from both places (postgres.c and rewriteHandler.c)

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Attachments:

0001-Separate-SQL-function-processing-from-postgres.c.patchapplication/octet-stream; name=0001-Separate-SQL-function-processing-from-postgres.c.patchDownload
From 895985d7d7178c25c0b9e24550c6a7e7fc7cd129 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Wed, 20 Oct 2010 16:02:07 -0300
Subject: [PATCH 1/2] Separate SQL function processing from postgres.c

Instead, simply inline the code from pg_parse_and_rewrite into the two
callers.  There's not enough gain for the modularity break.  Since this
makes pg_parse_and_rewrite() dead code, remove it.

Per comment from Tom.

Author: Marko Tiikkaja
---
 src/backend/catalog/pg_proc.c    |   24 +++++++++++++++++++--
 src/backend/executor/functions.c |   18 ++++++++++++++-
 src/backend/tcop/postgres.c      |   42 --------------------------------------
 src/include/tcop/tcopprot.h      |    2 -
 4 files changed, 37 insertions(+), 49 deletions(-)

diff --git a/src/backend/catalog/pg_proc.c b/src/backend/catalog/pg_proc.c
index 34cd862..a63fb9f 100644
--- a/src/backend/catalog/pg_proc.c
+++ b/src/backend/catalog/pg_proc.c
@@ -755,7 +755,9 @@ fmgr_sql_validator(PG_FUNCTION_ARGS)
 	Oid			funcoid = PG_GETARG_OID(0);
 	HeapTuple	tuple;
 	Form_pg_proc proc;
+	List	   *raw_parsetree_list;
 	List	   *querytree_list;
+	ListCell   *list_item;
 	bool		isnull;
 	Datum		tmp;
 	char	   *prosrc;
@@ -828,9 +830,25 @@ fmgr_sql_validator(PG_FUNCTION_ARGS)
 		 */
 		if (!haspolyarg)
 		{
-			querytree_list = pg_parse_and_rewrite(prosrc,
-												  proc->proargtypes.values,
-												  proc->pronargs);
+			/*
+			 * Parse and rewrite the queries in the function text.
+			 *
+			 * Even though check_sql_fn_retval is only interested in the last
+			 * query, we analyze all of them here to check for any errors.
+			 */
+			raw_parsetree_list = pg_parse_query(prosrc);
+			
+			querytree_list = NIL;
+			foreach(list_item, raw_parsetree_list)
+			{
+				Node *parsetree = (Node *) lfirst(list_item);
+
+				querytree_list = pg_analyze_and_rewrite(parsetree, prosrc,
+														proc->proargtypes.values, proc->pronargs);
+			}
+
+			Assert(querytree_list != NIL);
+
 			(void) check_sql_fn_retval(funcoid, proc->prorettype,
 									   querytree_list,
 									   NULL, NULL);
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 498bcba..46433d0 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -224,7 +224,9 @@ init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
 	SQLFunctionCachePtr fcache;
 	Oid		   *argOidVect;
 	int			nargs;
+	List	   *raw_parsetree_list;
 	List	   *queryTree_list;
+	ListCell   *list_item;
 	Datum		tmp;
 	bool		isNull;
 
@@ -319,7 +321,19 @@ init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
 	/*
 	 * Parse and rewrite the queries in the function text.
 	 */
-	queryTree_list = pg_parse_and_rewrite(fcache->src, argOidVect, nargs);
+	raw_parsetree_list = pg_parse_query(fcache->src);
+
+	queryTree_list = NIL;
+	foreach(list_item, raw_parsetree_list)
+	{
+		Node *parsetree = (Node *) lfirst(list_item);
+
+		queryTree_list = lappend(queryTree_list,
+								 pg_analyze_and_rewrite(parsetree,
+									 					fcache->src,
+														argOidVect,
+														nargs));
+	}
 
 	/*
 	 * Check that the function returns the type it claims to.  Although in
@@ -342,7 +356,7 @@ init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
 	 */
 	fcache->returnsTuple = check_sql_fn_retval(foid,
 											   rettype,
-											   queryTree_list,
+											   llast(queryTree_list),
 											   NULL,
 											   &fcache->junkFilter);
 
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index cba90a9..c6ce8de 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -505,48 +505,6 @@ client_read_ended(void)
 	}
 }
 
-
-/*
- * Parse a query string and pass it through the rewriter.
- *
- * A list of Query nodes is returned, since the string might contain
- * multiple queries and/or the rewriter might expand one query to several.
- *
- * NOTE: this routine is no longer used for processing interactive queries,
- * but it is still needed for parsing of SQL function bodies.
- */
-List *
-pg_parse_and_rewrite(const char *query_string,	/* string to execute */
-					 Oid *paramTypes,	/* parameter types */
-					 int numParams)		/* number of parameters */
-{
-	List	   *raw_parsetree_list;
-	List	   *querytree_list;
-	ListCell   *list_item;
-
-	/*
-	 * (1) parse the request string into a list of raw parse trees.
-	 */
-	raw_parsetree_list = pg_parse_query(query_string);
-
-	/*
-	 * (2) Do parse analysis and rule rewrite.
-	 */
-	querytree_list = NIL;
-	foreach(list_item, raw_parsetree_list)
-	{
-		Node	   *parsetree = (Node *) lfirst(list_item);
-
-		querytree_list = list_concat(querytree_list,
-									 pg_analyze_and_rewrite(parsetree,
-															query_string,
-															paramTypes,
-															numParams));
-	}
-
-	return querytree_list;
-}
-
 /*
  * Do raw parsing (only).
  *
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index f4026ec..0a9714d 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -45,8 +45,6 @@ typedef enum
 
 extern int	log_statement;
 
-extern List *pg_parse_and_rewrite(const char *query_string,
-					 Oid *paramTypes, int numParams);
 extern List *pg_parse_query(const char *query_string);
 extern List *pg_analyze_and_rewrite(Node *parsetree, const char *query_string,
 					   Oid *paramTypes, int numParams);
-- 
1.7.1

0002-The-remainder-of-Marko-s-patch.patchapplication/octet-stream; name=0002-The-remainder-of-Marko-s-patch.patchDownload
From 297de9f38c1b4b1dbdb7812081b6c8058aff6b33 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Wed, 20 Oct 2010 16:27:36 -0300
Subject: [PATCH 2/2] The remainder of Marko's patch

---
 src/backend/executor/functions.c |  261 ++++++++++++++++++++++++++------------
 src/backend/executor/spi.c       |   18 +++-
 src/backend/tcop/pquery.c        |   36 ++++-
 3 files changed, 223 insertions(+), 92 deletions(-)

diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 46433d0..6310fa1 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -90,18 +90,18 @@ typedef struct
 	ParamListInfo paramLI;		/* Param list representing current args */
 
 	Tuplestorestate *tstore;	/* where we accumulate result tuples */
+	Snapshot	snapshot;
 
 	JunkFilter *junkFilter;		/* will be NULL if function returns VOID */
 
-	/* head of linked list of execution_state records */
-	execution_state *func_state;
+	List *func_state;
 } SQLFunctionCache;
 
 typedef SQLFunctionCache *SQLFunctionCachePtr;
 
 
 /* non-export function prototypes */
-static execution_state *init_execution_state(List *queryTree_list,
+static List *init_execution_state(List *queryTree_list,
 					 SQLFunctionCachePtr fcache,
 					 bool lazyEvalOK);
 static void init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK);
@@ -123,61 +123,78 @@ static void sqlfunction_destroy(DestReceiver *self);
 
 
 /* Set up the list of per-query execution_state records for a SQL function */
-static execution_state *
+static List *
 init_execution_state(List *queryTree_list,
 					 SQLFunctionCachePtr fcache,
 					 bool lazyEvalOK)
 {
-	execution_state *firstes = NULL;
-	execution_state *preves = NULL;
+	execution_state *firstes;
+	execution_state *preves;
 	execution_state *lasttages = NULL;
-	ListCell   *qtl_item;
+	List	   *eslist;
+	ListCell   *lc1;
+	ListCell   *lc2;
+	List	   *qtlist;
+	Query	   *queryTree;
+
+
+	eslist = NIL;
 
-	foreach(qtl_item, queryTree_list)
+	foreach(lc1, queryTree_list)
 	{
-		Query	   *queryTree = (Query *) lfirst(qtl_item);
-		Node	   *stmt;
-		execution_state *newes;
+		qtlist = (List *) lfirst(lc1);
+		firstes = NULL;
+		preves = NULL;
 
-		Assert(IsA(queryTree, Query));
+		foreach(lc2, qtlist)
+		{
+			Node	   *stmt;
+			execution_state *newes;
 
-		if (queryTree->commandType == CMD_UTILITY)
-			stmt = queryTree->utilityStmt;
-		else
-			stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
+			queryTree = (Query *) lfirst(lc2);
 
-		/* Precheck all commands for validity in a function */
-		if (IsA(stmt, TransactionStmt))
-			ereport(ERROR,
-					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
-			/* translator: %s is a SQL statement name */
-					 errmsg("%s is not allowed in a SQL function",
-							CreateCommandTag(stmt))));
+			Assert(IsA(queryTree, Query));
 
-		if (fcache->readonly_func && !CommandIsReadOnly(stmt))
-			ereport(ERROR,
-					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
-			/* translator: %s is a SQL statement name */
-					 errmsg("%s is not allowed in a non-volatile function",
-							CreateCommandTag(stmt))));
+			if (queryTree->commandType == CMD_UTILITY)
+				stmt = queryTree->utilityStmt;
+			else
+				stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
 
-		newes = (execution_state *) palloc(sizeof(execution_state));
-		if (preves)
-			preves->next = newes;
-		else
-			firstes = newes;
+			/* Precheck all commands for validity in a function */
+			if (IsA(stmt, TransactionStmt))
+				ereport(ERROR,
+						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+				/* translator: %s is a SQL statement name */
+						 errmsg("%s is not allowed in a SQL function",
+								CreateCommandTag(stmt))));
 
-		newes->next = NULL;
-		newes->status = F_EXEC_START;
-		newes->setsResult = false;		/* might change below */
-		newes->lazyEval = false;	/* might change below */
-		newes->stmt = stmt;
-		newes->qd = NULL;
+			if (fcache->readonly_func && !CommandIsReadOnly(stmt))
+				ereport(ERROR,
+						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+				/* translator: %s is a SQL statement name */
+						 errmsg("%s is not allowed in a non-volatile function",
+								CreateCommandTag(stmt))));
+
+			newes = (execution_state *) palloc(sizeof(execution_state));
+			if (preves)
+				preves->next = newes;
+			else
+				firstes = newes;
 
-		if (queryTree->canSetTag)
-			lasttages = newes;
+			newes->next = NULL;
+			newes->status = F_EXEC_START;
+			newes->setsResult = false;		/* might change below */
+			newes->lazyEval = false;	/* might change below */
+			newes->stmt = stmt;
+			newes->qd = NULL;
 
-		preves = newes;
+			if (queryTree->canSetTag)
+				lasttages = newes;
+
+			preves = newes;
+		}
+
+		eslist = lappend(eslist, firstes);
 	}
 
 	/*
@@ -210,7 +227,7 @@ init_execution_state(List *queryTree_list,
 		}
 	}
 
-	return firstes;
+	return eslist;
 }
 
 /* Initialize the SQLFunctionCache for a SQL function */
@@ -388,24 +405,11 @@ init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
 static void
 postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
 {
-	Snapshot	snapshot;
 	DestReceiver *dest;
 
 	Assert(es->qd == NULL);
 
-	/*
-	 * In a read-only function, use the surrounding query's snapshot;
-	 * otherwise take a new snapshot for each query.  The snapshot should
-	 * include a fresh command ID so that all work to date in this transaction
-	 * is visible.
-	 */
-	if (fcache->readonly_func)
-		snapshot = GetActiveSnapshot();
-	else
-	{
-		CommandCounterIncrement();
-		snapshot = GetTransactionSnapshot();
-	}
+	Assert(ActiveSnapshotSet());
 
 	/*
 	 * If this query produces the function result, send its output to the
@@ -429,13 +433,14 @@ postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
 	if (IsA(es->stmt, PlannedStmt))
 		es->qd = CreateQueryDesc((PlannedStmt *) es->stmt,
 								 fcache->src,
-								 snapshot, InvalidSnapshot,
+								 GetActiveSnapshot(),
+								 InvalidSnapshot,
 								 dest,
 								 fcache->paramLI, 0);
 	else
 		es->qd = CreateUtilityQueryDesc(es->stmt,
 										fcache->src,
-										snapshot,
+										GetActiveSnapshot(),
 										dest,
 										fcache->paramLI);
 
@@ -631,6 +636,8 @@ fmgr_sql(PG_FUNCTION_ARGS)
 	execution_state *es;
 	TupleTableSlot *slot;
 	Datum		result;
+	List		*eslist;
+	ListCell    *eslc;
 
 	/*
 	 * Switch to context in which the fcache lives.  This ensures that
@@ -682,13 +689,13 @@ fmgr_sql(PG_FUNCTION_ARGS)
 		init_sql_fcache(fcinfo->flinfo, lazyEvalOK);
 		fcache = (SQLFunctionCachePtr) fcinfo->flinfo->fn_extra;
 	}
-	es = fcache->func_state;
+	eslist = fcache->func_state;
 
 	/*
 	 * Convert params to appropriate format if starting a fresh execution. (If
 	 * continuing execution, we can re-use prior params.)
 	 */
-	if (es && es->status == F_EXEC_START)
+	if (linitial(eslist) && ((execution_state *) linitial(eslist))->status == F_EXEC_START)
 		postquel_sub_params(fcache, fcinfo);
 
 	/*
@@ -701,8 +708,18 @@ fmgr_sql(PG_FUNCTION_ARGS)
 	/*
 	 * Find first unfinished query in function.
 	 */
-	while (es && es->status == F_EXEC_DONE)
-		es = es->next;
+
+	es = NULL; /* keep compiler quiet */
+	foreach(eslc, eslist)
+	{
+		es = (execution_state *) lfirst(eslc);
+
+		while (es && es->status == F_EXEC_DONE)
+			es = es->next;
+
+		if (es)
+			break;
+	}
 
 	/*
 	 * Execute each command in the function one after another until we either
@@ -713,8 +730,31 @@ fmgr_sql(PG_FUNCTION_ARGS)
 		bool		completed;
 
 		if (es->status == F_EXEC_START)
+		{
+			if (!fcache->readonly_func)
+			{
+				/*
+				 * In a read-only function, use the surrounding query's snapshot;
+				 * otherwise take a new snapshot if we don't have one yet.  The
+				 * snapshot should include a fresh command ID so that all work to
+				 * date in this transaction is visible.
+				 */
+				if (!fcache->snapshot)
+				{
+					CommandCounterIncrement();
+					fcache->snapshot = RegisterSnapshot(GetTransactionSnapshot());
+					PushActiveSnapshot(fcache->snapshot);
+				}
+				else
+					PushUpdatedSnapshot(fcache->snapshot);
+			}
+			
 			postquel_start(es, fcache);
 
+			if (!fcache->readonly_func)
+				PopActiveSnapshot();
+		}
+
 		completed = postquel_getnext(es, fcache);
 
 		/*
@@ -740,6 +780,25 @@ fmgr_sql(PG_FUNCTION_ARGS)
 		if (es->status != F_EXEC_DONE)
 			break;
 		es = es->next;
+
+		if (!es)
+		{
+			eslc = lnext(eslc);
+			if (!eslc)
+				break;
+
+			es = (execution_state *) lfirst(eslc);
+
+			/* make sure we take a new snapshot for this query list */
+			if (!fcache->readonly_func)
+			{
+				Assert(fcache->snapshot != InvalidSnapshot);
+				UnregisterSnapshot(fcache->snapshot);
+				fcache->snapshot = InvalidSnapshot;
+			}
+			else
+				Assert(fcache->snapshot == InvalidSnapshot);
+		}
 	}
 
 	/*
@@ -808,6 +867,11 @@ fmgr_sql(PG_FUNCTION_ARGS)
 											  PointerGetDatum(fcache));
 				fcache->shutdown_reg = false;
 			}
+
+			/* Unregister snapshot if we have one */
+			if (fcache->snapshot != InvalidSnapshot)
+				UnregisterSnapshot(fcache->snapshot);
+			fcache->snapshot = InvalidSnapshot;
 		}
 		else
 		{
@@ -834,6 +898,11 @@ fmgr_sql(PG_FUNCTION_ARGS)
 											  PointerGetDatum(fcache));
 				fcache->shutdown_reg = false;
 			}
+
+			/* Unregister snapshot if we have one */
+			if (fcache->snapshot != InvalidSnapshot)
+				UnregisterSnapshot(fcache->snapshot);
+			fcache->snapshot = InvalidSnapshot;
 		}
 	}
 	else
@@ -864,6 +933,11 @@ fmgr_sql(PG_FUNCTION_ARGS)
 
 		/* Clear the tuplestore, but keep it for next time */
 		tuplestore_clear(fcache->tstore);
+
+		/* Unregister snapshot if we have one */
+		if (fcache->snapshot != InvalidSnapshot)
+			UnregisterSnapshot(fcache->snapshot);
+		fcache->snapshot = InvalidSnapshot;
 	}
 
 	/*
@@ -872,11 +946,14 @@ fmgr_sql(PG_FUNCTION_ARGS)
 	 */
 	if (es == NULL)
 	{
-		es = fcache->func_state;
-		while (es)
+		foreach(eslc, fcache->func_state)
 		{
-			es->status = F_EXEC_START;
-			es = es->next;
+			es = (execution_state *) lfirst(eslc);
+			while (es)
+			{
+				es->status = F_EXEC_START;
+				es = es->next;
+			}
 		}
 	}
 
@@ -927,19 +1004,24 @@ sql_exec_error_callback(void *arg)
 	{
 		execution_state *es;
 		int			query_num;
+		ListCell   *lc;
 
-		es = fcache->func_state;
+		es = NULL; /* keep compiler quiet */
 		query_num = 1;
-		while (es)
+		foreach(lc, fcache->func_state)
 		{
-			if (es->qd)
+			es = (execution_state *) lfirst(lc);
+			while (es)
 			{
-				errcontext("SQL function \"%s\" statement %d",
-						   fcache->fname, query_num);
-				break;
+				if (es->qd)
+				{
+					errcontext("SQL function \"%s\" statement %d",
+							   fcache->fname, query_num);
+					break;
+				}
+				es = es->next;
+				query_num++;
 			}
-			es = es->next;
-			query_num++;
 		}
 		if (es == NULL)
 		{
@@ -970,18 +1052,29 @@ static void
 ShutdownSQLFunction(Datum arg)
 {
 	SQLFunctionCachePtr fcache = (SQLFunctionCachePtr) DatumGetPointer(arg);
-	execution_state *es = fcache->func_state;
+	execution_state *es;
+	ListCell		*lc;
 
-	while (es != NULL)
+	foreach(lc, fcache->func_state)
 	{
-		/* Shut down anything still running */
-		if (es->status == F_EXEC_RUN)
-			postquel_end(es);
-		/* Reset states to START in case we're called again */
-		es->status = F_EXEC_START;
-		es = es->next;
+		es = (execution_state *) lfirst(lc);
+
+		while (es)
+		{
+			/* Shut down anything still running */
+			if (es->status == F_EXEC_RUN)
+				postquel_end(es);
+			/* Reset states to START in case we're called again */
+			es->status = F_EXEC_START;
+			es = es->next;
+		}
 	}
 
+	/* Unregister snapshot if we have one */
+	if (fcache->snapshot != InvalidSnapshot)
+		UnregisterSnapshot(fcache->snapshot);
+	fcache->snapshot = InvalidSnapshot;
+
 	/* Release tuplestore if we have one */
 	if (fcache->tstore)
 		tuplestore_end(fcache->tstore);
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index c579017..2998b3a 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -1769,6 +1769,7 @@ _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
 	SPITupleTable *my_tuptable = NULL;
 	int			res = 0;
 	bool		have_active_snap = ActiveSnapshotSet();
+	bool		registered_snap = false;
 	ErrorContextCallback spierrcontext;
 	CachedPlan *cplan = NULL;
 	ListCell   *lc1;
@@ -1872,8 +1873,10 @@ _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
 				}
 				else
 				{
-					PushActiveSnapshot(GetTransactionSnapshot());
+					snapshot = RegisterSnapshot(GetTransactionSnapshot());
+					PushActiveSnapshot(snapshot);
 					pushed_active_snap = true;
+					registered_snap = true;
 				}
 			}
 			else
@@ -1966,10 +1969,23 @@ _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
 		 */
 		if (!read_only)
 			CommandCounterIncrement();
+
+		/*
+		 * If we took a new snapshot for this query list, unregister it and
+		 * make sure we take a new one for the next list.
+		 */
+		if (registered_snap)
+		{
+			UnregisterSnapshot(snapshot);
+			snapshot = InvalidSnapshot;
+		}
 	}
 
 fail:
 
+	if (registered_snap)
+		UnregisterSnapshot(snapshot);
+
 	/* We no longer need the cached plan refcount, if any */
 	if (cplan)
 		ReleaseCachedPlan(cplan, true);
diff --git a/src/backend/tcop/pquery.c b/src/backend/tcop/pquery.c
index 8eb02da..b475e11 100644
--- a/src/backend/tcop/pquery.c
+++ b/src/backend/tcop/pquery.c
@@ -170,11 +170,6 @@ ProcessQuery(PlannedStmt *plan,
 	elog(DEBUG3, "ProcessQuery");
 
 	/*
-	 * Must always set a snapshot for plannable queries.
-	 */
-	PushActiveSnapshot(GetTransactionSnapshot());
-
-	/*
 	 * Create the QueryDesc object
 	 */
 	queryDesc = CreateQueryDesc(plan, sourceText,
@@ -234,8 +229,6 @@ ProcessQuery(PlannedStmt *plan,
 	/* Now take care of any queued AFTER triggers */
 	AfterTriggerEndQuery(queryDesc->estate);
 
-	PopActiveSnapshot();
-
 	/*
 	 * Now, we close down all the scans and free allocated resources.
 	 */
@@ -1220,6 +1213,7 @@ PortalRunMulti(Portal portal, bool isTopLevel,
 			   char *completionTag)
 {
 	ListCell   *stmtlist_item;
+	Snapshot	snapshot = InvalidSnapshot;
 
 	/*
 	 * If the destination is DestRemoteExecute, change to DestNone.  The
@@ -1262,6 +1256,15 @@ PortalRunMulti(Portal portal, bool isTopLevel,
 			if (log_executor_stats)
 				ResetUsage();
 
+			/* if no snapshot is set, grab a new one and register it */
+			if (snapshot == InvalidSnapshot)
+			{
+				snapshot = RegisterSnapshot(GetTransactionSnapshot());
+				PushActiveSnapshot(snapshot);
+			}
+			else
+				PushUpdatedSnapshot(snapshot);
+
 			if (pstmt->canSetTag)
 			{
 				/* statement can set tag string */
@@ -1279,6 +1282,8 @@ PortalRunMulti(Portal portal, bool isTopLevel,
 							 altdest, NULL);
 			}
 
+			PopActiveSnapshot();
+
 			if (log_executor_stats)
 				ShowUsage("EXECUTOR STATISTICS");
 
@@ -1291,11 +1296,25 @@ PortalRunMulti(Portal portal, bool isTopLevel,
 			 *
 			 * These are assumed canSetTag if they're the only stmt in the
 			 * portal.
+			 *
+			 * NotifyStmt is the only utility statement allowed in a list of
+			 * rewritten queries, and it doesn't need a snapshot so we don't
+			 * need to worry about it.  However, if the list has only one
+			 * statement and it's a utility statement, we are not allowed to
+			 * take a snapshot.  See the first comment in PortalRunUtility().
 			 */
 			if (list_length(portal->stmts) == 1)
+			{
+				Assert(snapshot == InvalidSnapshot);
+
 				PortalRunUtility(portal, stmt, isTopLevel, dest, completionTag);
+			}
 			else
+			{
+				Assert(IsA(stmt, NotifyStmt));
+
 				PortalRunUtility(portal, stmt, isTopLevel, altdest, NULL);
+			}
 		}
 
 		/*
@@ -1313,6 +1332,9 @@ PortalRunMulti(Portal portal, bool isTopLevel,
 		MemoryContextDeleteChildren(PortalGetHeapMemory(portal));
 	}
 
+	if (snapshot != InvalidSnapshot)
+		UnregisterSnapshot(snapshot);
+
 	/*
 	 * If a command completion tag was supplied, use it.  Otherwise use the
 	 * portal's commandTag as the default completion tag.
-- 
1.7.1

#17Alvaro Herrera
alvherre@commandprompt.com
In reply to: Alvaro Herrera (#16)
Re: Review: Fix snapshot taking inconsistencies

Excerpts from Alvaro Herrera's message of mié oct 20 16:33:12 -0300 2010:

The only quarrel I have with this code shuffling is that
pg_rewrite_query is being called from exec_parse_message. Since it's
now a static function, it would have to stop being static so that it can
be called from both places (postgres.c and rewriteHandler.c)

Actually, I just noticed that the "remainder" patch uses pg_plan_query,
which is also in postgres.c. This function along with its sibling
pg_plan_queries is also called from other internal places, like the
PREPARE code, SPI and the plan cache.

It strikes me that if we really want to restructure things to divide
client interaction from other query-processing routines, we should
create another file, say src/backend/tcop/queries.c; this would have
stuff like pg_plan_query, pg_plan_queries, pg_rewrite_query, and the
other things that the patch was evicting from postgres.c (plus, I
imagine, a bunch of other stuff that I may be missing). In fact, if we
go down this route, there would be no point in removing
pg_parse_and_rewrite; we would just move it to this new module.

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

#18Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#17)
Re: Review: Fix snapshot taking inconsistencies

Alvaro Herrera <alvherre@commandprompt.com> writes:

It strikes me that if we really want to restructure things to divide
client interaction from other query-processing routines, we should
create another file, say src/backend/tcop/queries.c; this would have
stuff like pg_plan_query, pg_plan_queries, pg_rewrite_query, and the
other things that the patch was evicting from postgres.c (plus, I
imagine, a bunch of other stuff that I may be missing). In fact, if we
go down this route, there would be no point in removing
pg_parse_and_rewrite; we would just move it to this new module.

Yeah, possibly that would be a good idea.

To my mind, the first thing that has to be resolved before messing
around in this area is whether or not we want the logging/statistics
behavior of these functions to apply when they are used in contexts
other than interactive queries. Personally I would vote no, mainly
because I don't think that behavior is very sensible in nested
execution. If that is the decision, then probably these functions
should stay where they are and as they are, and we just deprecate
outside use of them. I'm not sure whether there's enough left after
deleting the logging/statistics behavior to justify making exported
versions, as opposed to just having the callers call the next-layer-down
functionality directly.

regards, tom lane

#19Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Tom Lane (#14)
1 attachment(s)
Re: Review: Fix snapshot taking inconsistencies

Hi,

Here's an updated patch.

I'm still not too fond of the logic in spi.c, but I can't see a better
way to do this. If someone sees a better way, I'm not going to object.

I also made some changes to the SQL functions now that we have a
different API. The previous code pushed and popped snapshots quite heavily.

I'd also like to see more regression tests for SQL functions, but I'm
going to submit my suggestions as a separate patch.

Regards,
Marko Tiikkaja

Attachments:

snapshot4.patchtext/plain; charset=iso-8859-1; name=snapshot4.patch; x-mac-creator=0; x-mac-type=0Download
*** a/src/backend/catalog/pg_proc.c
--- b/src/backend/catalog/pg_proc.c
***************
*** 755,761 **** fmgr_sql_validator(PG_FUNCTION_ARGS)
--- 755,763 ----
  	Oid			funcoid = PG_GETARG_OID(0);
  	HeapTuple	tuple;
  	Form_pg_proc proc;
+ 	List	   *raw_parsetree_list;
  	List	   *querytree_list;
+ 	ListCell   *list_item;
  	bool		isnull;
  	Datum		tmp;
  	char	   *prosrc;
***************
*** 828,836 **** fmgr_sql_validator(PG_FUNCTION_ARGS)
  		 */
  		if (!haspolyarg)
  		{
! 			querytree_list = pg_parse_and_rewrite(prosrc,
! 												  proc->proargtypes.values,
! 												  proc->pronargs);
  			(void) check_sql_fn_retval(funcoid, proc->prorettype,
  									   querytree_list,
  									   NULL, NULL);
--- 830,854 ----
  		 */
  		if (!haspolyarg)
  		{
! 			/*
! 			 * Parse and rewrite the queries in the function text.
! 			 *
! 			 * Even though check_sql_fn_retval is only interested in the last
! 			 * query, we analyze all of them here to check for any errors.
! 			 */
! 			raw_parsetree_list = pg_parse_query(prosrc);
! 			
! 			querytree_list = NIL;
! 			foreach(list_item, raw_parsetree_list)
! 			{
! 				Node *parsetree = (Node *) lfirst(list_item);
! 
! 				querytree_list = pg_analyze_and_rewrite(parsetree, prosrc,
! 														proc->proargtypes.values, proc->pronargs);
! 			}
! 
! 			Assert(querytree_list != NIL);
! 
  			(void) check_sql_fn_retval(funcoid, proc->prorettype,
  									   querytree_list,
  									   NULL, NULL);
*** a/src/backend/executor/functions.c
--- b/src/backend/executor/functions.c
***************
*** 84,89 **** typedef struct
--- 84,90 ----
  	bool		returnsSet;		/* true if returning multiple rows */
  	bool		returnsTuple;	/* true if returning whole tuple result */
  	bool		shutdown_reg;	/* true if registered shutdown callback */
+ 	bool		snapshot;		/* true if pushed an active snapshot */
  	bool		readonly_func;	/* true to run in "read only" mode */
  	bool		lazyEval;		/* true if using lazyEval for result query */
  
***************
*** 93,107 **** typedef struct
  
  	JunkFilter *junkFilter;		/* will be NULL if function returns VOID */
  
! 	/* head of linked list of execution_state records */
! 	execution_state *func_state;
  } SQLFunctionCache;
  
  typedef SQLFunctionCache *SQLFunctionCachePtr;
  
  
  /* non-export function prototypes */
! static execution_state *init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK);
  static void init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK);
--- 94,107 ----
  
  	JunkFilter *junkFilter;		/* will be NULL if function returns VOID */
  
! 	List *func_state;
  } SQLFunctionCache;
  
  typedef SQLFunctionCache *SQLFunctionCachePtr;
  
  
  /* non-export function prototypes */
! static List *init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK);
  static void init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK);
***************
*** 123,183 **** static void sqlfunction_destroy(DestReceiver *self);
  
  
  /* Set up the list of per-query execution_state records for a SQL function */
! static execution_state *
  init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK)
  {
! 	execution_state *firstes = NULL;
! 	execution_state *preves = NULL;
  	execution_state *lasttages = NULL;
! 	ListCell   *qtl_item;
  
! 	foreach(qtl_item, queryTree_list)
  	{
! 		Query	   *queryTree = (Query *) lfirst(qtl_item);
! 		Node	   *stmt;
! 		execution_state *newes;
  
! 		Assert(IsA(queryTree, Query));
  
! 		if (queryTree->commandType == CMD_UTILITY)
! 			stmt = queryTree->utilityStmt;
! 		else
! 			stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
  
! 		/* Precheck all commands for validity in a function */
! 		if (IsA(stmt, TransactionStmt))
! 			ereport(ERROR,
! 					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 			/* translator: %s is a SQL statement name */
! 					 errmsg("%s is not allowed in a SQL function",
! 							CreateCommandTag(stmt))));
  
! 		if (fcache->readonly_func && !CommandIsReadOnly(stmt))
! 			ereport(ERROR,
! 					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 			/* translator: %s is a SQL statement name */
! 					 errmsg("%s is not allowed in a non-volatile function",
! 							CreateCommandTag(stmt))));
  
! 		newes = (execution_state *) palloc(sizeof(execution_state));
! 		if (preves)
! 			preves->next = newes;
! 		else
! 			firstes = newes;
  
! 		newes->next = NULL;
! 		newes->status = F_EXEC_START;
! 		newes->setsResult = false;		/* might change below */
! 		newes->lazyEval = false;	/* might change below */
! 		newes->stmt = stmt;
! 		newes->qd = NULL;
  
! 		if (queryTree->canSetTag)
! 			lasttages = newes;
  
! 		preves = newes;
  	}
  
  	/*
--- 123,200 ----
  
  
  /* Set up the list of per-query execution_state records for a SQL function */
! static List *
  init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK)
  {
! 	execution_state *firstes;
! 	execution_state *preves;
  	execution_state *lasttages = NULL;
! 	List	   *eslist;
! 	ListCell   *lc1;
! 	ListCell   *lc2;
! 	List	   *qtlist;
! 	Query	   *queryTree;
  
! 
! 	eslist = NIL;
! 
! 	foreach(lc1, queryTree_list)
  	{
! 		qtlist = (List *) lfirst(lc1);
! 		firstes = NULL;
! 		preves = NULL;
  
! 		foreach(lc2, qtlist)
! 		{
! 			Node	   *stmt;
! 			execution_state *newes;
  
! 			queryTree = (Query *) lfirst(lc2);
  
! 			Assert(IsA(queryTree, Query));
  
! 			if (queryTree->commandType == CMD_UTILITY)
! 				stmt = queryTree->utilityStmt;
! 			else
! 				stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
  
! 			/* Precheck all commands for validity in a function */
! 			if (IsA(stmt, TransactionStmt))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 				/* translator: %s is a SQL statement name */
! 						 errmsg("%s is not allowed in a SQL function",
! 								CreateCommandTag(stmt))));
! 
! 			if (fcache->readonly_func && !CommandIsReadOnly(stmt))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 				/* translator: %s is a SQL statement name */
! 						 errmsg("%s is not allowed in a non-volatile function",
! 								CreateCommandTag(stmt))));
! 
! 			newes = (execution_state *) palloc(sizeof(execution_state));
! 			if (preves)
! 				preves->next = newes;
! 			else
! 				firstes = newes;
! 
! 			newes->next = NULL;
! 			newes->status = F_EXEC_START;
! 			newes->setsResult = false;		/* might change below */
! 			newes->lazyEval = false;	/* might change below */
! 			newes->stmt = stmt;
! 			newes->qd = NULL;
  
! 			if (queryTree->canSetTag)
! 				lasttages = newes;
  
! 			preves = newes;
! 		}
  
! 		eslist = lappend(eslist, firstes);
  	}
  
  	/*
***************
*** 210,216 **** init_execution_state(List *queryTree_list,
  		}
  	}
  
! 	return firstes;
  }
  
  /* Initialize the SQLFunctionCache for a SQL function */
--- 227,233 ----
  		}
  	}
  
! 	return eslist;
  }
  
  /* Initialize the SQLFunctionCache for a SQL function */
***************
*** 224,230 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
--- 241,249 ----
  	SQLFunctionCachePtr fcache;
  	Oid		   *argOidVect;
  	int			nargs;
+ 	List	   *raw_parsetree_list;
  	List	   *queryTree_list;
+ 	ListCell   *list_item;
  	Datum		tmp;
  	bool		isNull;
  
***************
*** 319,325 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  	/*
  	 * Parse and rewrite the queries in the function text.
  	 */
! 	queryTree_list = pg_parse_and_rewrite(fcache->src, argOidVect, nargs);
  
  	/*
  	 * Check that the function returns the type it claims to.  Although in
--- 338,356 ----
  	/*
  	 * Parse and rewrite the queries in the function text.
  	 */
! 	raw_parsetree_list = pg_parse_query(fcache->src);
! 
! 	queryTree_list = NIL;
! 	foreach(list_item, raw_parsetree_list)
! 	{
! 		Node *parsetree = (Node *) lfirst(list_item);
! 
! 		queryTree_list = lappend(queryTree_list,
! 								 pg_analyze_and_rewrite(parsetree,
! 									 					fcache->src,
! 														argOidVect,
! 														nargs));
! 	}
  
  	/*
  	 * Check that the function returns the type it claims to.  Although in
***************
*** 342,348 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  	 */
  	fcache->returnsTuple = check_sql_fn_retval(foid,
  											   rettype,
! 											   queryTree_list,
  											   NULL,
  											   &fcache->junkFilter);
  
--- 373,379 ----
  	 */
  	fcache->returnsTuple = check_sql_fn_retval(foid,
  											   rettype,
! 											   llast(queryTree_list),
  											   NULL,
  											   &fcache->junkFilter);
  
***************
*** 374,397 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  static void
  postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  {
- 	Snapshot	snapshot;
  	DestReceiver *dest;
  
  	Assert(es->qd == NULL);
  
! 	/*
! 	 * In a read-only function, use the surrounding query's snapshot;
! 	 * otherwise take a new snapshot for each query.  The snapshot should
! 	 * include a fresh command ID so that all work to date in this transaction
! 	 * is visible.
! 	 */
! 	if (fcache->readonly_func)
! 		snapshot = GetActiveSnapshot();
! 	else
! 	{
! 		CommandCounterIncrement();
! 		snapshot = GetTransactionSnapshot();
! 	}
  
  	/*
  	 * If this query produces the function result, send its output to the
--- 405,415 ----
  static void
  postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  {
  	DestReceiver *dest;
  
  	Assert(es->qd == NULL);
  
! 	Assert(ActiveSnapshotSet());
  
  	/*
  	 * If this query produces the function result, send its output to the
***************
*** 415,427 **** postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  	if (IsA(es->stmt, PlannedStmt))
  		es->qd = CreateQueryDesc((PlannedStmt *) es->stmt,
  								 fcache->src,
! 								 snapshot, InvalidSnapshot,
  								 dest,
  								 fcache->paramLI, 0);
  	else
  		es->qd = CreateUtilityQueryDesc(es->stmt,
  										fcache->src,
! 										snapshot,
  										dest,
  										fcache->paramLI);
  
--- 433,446 ----
  	if (IsA(es->stmt, PlannedStmt))
  		es->qd = CreateQueryDesc((PlannedStmt *) es->stmt,
  								 fcache->src,
! 								 GetActiveSnapshot(),
! 								 InvalidSnapshot,
  								 dest,
  								 fcache->paramLI, 0);
  	else
  		es->qd = CreateUtilityQueryDesc(es->stmt,
  										fcache->src,
! 										GetActiveSnapshot(),
  										dest,
  										fcache->paramLI);
  
***************
*** 451,459 **** postquel_getnext(execution_state *es, SQLFunctionCachePtr fcache)
  {
  	bool		result;
  
- 	/* Make our snapshot the active one for any called functions */
- 	PushActiveSnapshot(es->qd->snapshot);
- 
  	if (es->qd->utilitystmt)
  	{
  		/* ProcessUtility needs the PlannedStmt for DECLARE CURSOR */
--- 470,475 ----
***************
*** 481,488 **** postquel_getnext(execution_state *es, SQLFunctionCachePtr fcache)
  		result = (count == 0L || es->qd->estate->es_processed == 0);
  	}
  
- 	PopActiveSnapshot();
- 
  	return result;
  }
  
--- 497,502 ----
***************
*** 496,509 **** postquel_end(execution_state *es)
  	/* Utility commands don't need Executor. */
  	if (es->qd->utilitystmt == NULL)
  	{
- 		/* Make our snapshot the active one for any called functions */
- 		PushActiveSnapshot(es->qd->snapshot);
- 
  		if (es->qd->operation != CMD_SELECT)
  			AfterTriggerEndQuery(es->qd->estate);
  		ExecutorEnd(es->qd);
- 
- 		PopActiveSnapshot();
  	}
  
  	(*es->qd->dest->rDestroy) (es->qd->dest);
--- 510,518 ----
***************
*** 617,622 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 626,633 ----
  	execution_state *es;
  	TupleTableSlot *slot;
  	Datum		result;
+ 	List		*eslist;
+ 	ListCell    *eslc;
  
  	/*
  	 * Switch to context in which the fcache lives.  This ensures that
***************
*** 668,680 **** fmgr_sql(PG_FUNCTION_ARGS)
  		init_sql_fcache(fcinfo->flinfo, lazyEvalOK);
  		fcache = (SQLFunctionCachePtr) fcinfo->flinfo->fn_extra;
  	}
! 	es = fcache->func_state;
  
  	/*
  	 * Convert params to appropriate format if starting a fresh execution. (If
  	 * continuing execution, we can re-use prior params.)
  	 */
! 	if (es && es->status == F_EXEC_START)
  		postquel_sub_params(fcache, fcinfo);
  
  	/*
--- 679,691 ----
  		init_sql_fcache(fcinfo->flinfo, lazyEvalOK);
  		fcache = (SQLFunctionCachePtr) fcinfo->flinfo->fn_extra;
  	}
! 	eslist = fcache->func_state;
  
  	/*
  	 * Convert params to appropriate format if starting a fresh execution. (If
  	 * continuing execution, we can re-use prior params.)
  	 */
! 	if (linitial(eslist) && ((execution_state *) linitial(eslist))->status == F_EXEC_START)
  		postquel_sub_params(fcache, fcinfo);
  
  	/*
***************
*** 687,694 **** fmgr_sql(PG_FUNCTION_ARGS)
  	/*
  	 * Find first unfinished query in function.
  	 */
! 	while (es && es->status == F_EXEC_DONE)
! 		es = es->next;
  
  	/*
  	 * Execute each command in the function one after another until we either
--- 698,715 ----
  	/*
  	 * Find first unfinished query in function.
  	 */
! 
! 	es = NULL; /* keep compiler quiet */
! 	foreach(eslc, eslist)
! 	{
! 		es = (execution_state *) lfirst(eslc);
! 
! 		while (es && es->status == F_EXEC_DONE)
! 			es = es->next;
! 
! 		if (es)
! 			break;
! 	}
  
  	/*
  	 * Execute each command in the function one after another until we either
***************
*** 699,705 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 720,746 ----
  		bool		completed;
  
  		if (es->status == F_EXEC_START)
+ 		{
+ 			if (!fcache->readonly_func)
+ 			{
+ 				/*
+ 				 * In a read-only function, use the surrounding query's snapshot;
+ 				 * otherwise take a new snapshot if we don't have one yet.  The
+ 				 * snapshot should include a fresh command ID so that all work to
+ 				 * date in this transaction is visible.
+ 				 */
+ 				if (!fcache->snapshot)
+ 				{
+ 					CommandCounterIncrement();
+ 					PushActiveSnapshot(GetTransactionSnapshot());
+ 					fcache->snapshot = true;
+ 				}
+ 				else
+ 					UpdateActiveSnapshot();
+ 			}
+ 			
  			postquel_start(es, fcache);
+ 		}
  
  		completed = postquel_getnext(es, fcache);
  
***************
*** 726,731 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 767,791 ----
  		if (es->status != F_EXEC_DONE)
  			break;
  		es = es->next;
+ 
+ 		if (!es)
+ 		{
+ 			eslc = lnext(eslc);
+ 			if (!eslc)
+ 				break;
+ 
+ 			es = (execution_state *) lfirst(eslc);
+ 
+ 			/* make sure we take a new snapshot for this query list */
+ 			if (!fcache->readonly_func)
+ 			{
+ 				Assert(fcache->snapshot);
+ 				PopActiveSnapshot();
+ 				fcache->snapshot = false;
+ 			}
+ 			else
+ 				Assert(!fcache->snapshot);
+ 		}
  	}
  
  	/*
***************
*** 794,799 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 854,864 ----
  											  PointerGetDatum(fcache));
  				fcache->shutdown_reg = false;
  			}
+ 
+ 			/* Release snapshot if we have one */
+ 			if (fcache->snapshot)
+ 				PopActiveSnapshot();
+ 			fcache->snapshot = false;
  		}
  		else
  		{
***************
*** 820,825 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 885,895 ----
  											  PointerGetDatum(fcache));
  				fcache->shutdown_reg = false;
  			}
+ 
+ 			/* Release snapshot if we have one */
+ 			if (fcache->snapshot)
+ 				PopActiveSnapshot();
+ 			fcache->snapshot = false;
  		}
  	}
  	else
***************
*** 850,855 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 920,930 ----
  
  		/* Clear the tuplestore, but keep it for next time */
  		tuplestore_clear(fcache->tstore);
+ 
+ 		/* Release snapshot if we have one */
+ 		if (fcache->snapshot)
+ 			PopActiveSnapshot();
+ 		fcache->snapshot = false;
  	}
  
  	/*
***************
*** 858,868 **** fmgr_sql(PG_FUNCTION_ARGS)
  	 */
  	if (es == NULL)
  	{
! 		es = fcache->func_state;
! 		while (es)
  		{
! 			es->status = F_EXEC_START;
! 			es = es->next;
  		}
  	}
  
--- 933,946 ----
  	 */
  	if (es == NULL)
  	{
! 		foreach(eslc, fcache->func_state)
  		{
! 			es = (execution_state *) lfirst(eslc);
! 			while (es)
! 			{
! 				es->status = F_EXEC_START;
! 				es = es->next;
! 			}
  		}
  	}
  
***************
*** 913,931 **** sql_exec_error_callback(void *arg)
  	{
  		execution_state *es;
  		int			query_num;
  
! 		es = fcache->func_state;
  		query_num = 1;
! 		while (es)
  		{
! 			if (es->qd)
  			{
! 				errcontext("SQL function \"%s\" statement %d",
! 						   fcache->fname, query_num);
! 				break;
  			}
- 			es = es->next;
- 			query_num++;
  		}
  		if (es == NULL)
  		{
--- 991,1014 ----
  	{
  		execution_state *es;
  		int			query_num;
+ 		ListCell   *lc;
  
! 		es = NULL; /* keep compiler quiet */
  		query_num = 1;
! 		foreach(lc, fcache->func_state)
  		{
! 			es = (execution_state *) lfirst(lc);
! 			while (es)
  			{
! 				if (es->qd)
! 				{
! 					errcontext("SQL function \"%s\" statement %d",
! 							   fcache->fname, query_num);
! 					break;
! 				}
! 				es = es->next;
! 				query_num++;
  			}
  		}
  		if (es == NULL)
  		{
***************
*** 956,973 **** static void
  ShutdownSQLFunction(Datum arg)
  {
  	SQLFunctionCachePtr fcache = (SQLFunctionCachePtr) DatumGetPointer(arg);
! 	execution_state *es = fcache->func_state;
  
! 	while (es != NULL)
  	{
! 		/* Shut down anything still running */
! 		if (es->status == F_EXEC_RUN)
! 			postquel_end(es);
! 		/* Reset states to START in case we're called again */
! 		es->status = F_EXEC_START;
! 		es = es->next;
  	}
  
  	/* Release tuplestore if we have one */
  	if (fcache->tstore)
  		tuplestore_end(fcache->tstore);
--- 1039,1067 ----
  ShutdownSQLFunction(Datum arg)
  {
  	SQLFunctionCachePtr fcache = (SQLFunctionCachePtr) DatumGetPointer(arg);
! 	execution_state *es;
! 	ListCell		*lc;
  
! 	foreach(lc, fcache->func_state)
  	{
! 		es = (execution_state *) lfirst(lc);
! 
! 		while (es)
! 		{
! 			/* Shut down anything still running */
! 			if (es->status == F_EXEC_RUN)
! 				postquel_end(es);
! 			/* Reset states to START in case we're called again */
! 			es->status = F_EXEC_START;
! 			es = es->next;
! 		}
  	}
  
+ 	/* Release snapshot if we have one */
+ 	if (fcache->snapshot)
+ 		PopActiveSnapshot();
+ 	fcache->snapshot = false;
+ 
  	/* Release tuplestore if we have one */
  	if (fcache->tstore)
  		tuplestore_end(fcache->tstore);
*** a/src/backend/executor/spi.c
--- b/src/backend/executor/spi.c
***************
*** 1768,1774 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
  	Oid			my_lastoid = InvalidOid;
  	SPITupleTable *my_tuptable = NULL;
  	int			res = 0;
! 	bool		have_active_snap = ActiveSnapshotSet();
  	ErrorContextCallback spierrcontext;
  	CachedPlan *cplan = NULL;
  	ListCell   *lc1;
--- 1768,1774 ----
  	Oid			my_lastoid = InvalidOid;
  	SPITupleTable *my_tuptable = NULL;
  	int			res = 0;
! 	bool		use_single_snap;
  	ErrorContextCallback spierrcontext;
  	CachedPlan *cplan = NULL;
  	ListCell   *lc1;
***************
*** 1781,1786 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
--- 1781,1798 ----
  	spierrcontext.previous = error_context_stack;
  	error_context_stack = &spierrcontext;
  
+ 	if (read_only || snapshot != InvalidSnapshot)
+ 	{
+ 		if (snapshot != InvalidSnapshot)
+ 			PushActiveSnapshot(snapshot);
+ 		else if (read_only && ActiveSnapshotSet())
+ 			PushActiveSnapshot(GetActiveSnapshot());
+ 
+ 		use_single_snap = true;
+ 	}
+ 	else
+ 		use_single_snap = false;
+ 
  	foreach(lc1, plan->plancache_list)
  	{
  		CachedPlanSource *plansource = (CachedPlanSource *) lfirst(lc1);
***************
*** 1802,1813 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
  			stmt_list = plansource->plan->stmt_list;
  		}
  
  		foreach(lc2, stmt_list)
  		{
  			Node	   *stmt = (Node *) lfirst(lc2);
  			bool		canSetTag;
  			DestReceiver *dest;
- 			bool		pushed_active_snap = false;
  
  			_SPI_current->processed = 0;
  			_SPI_current->lastoid = InvalidOid;
--- 1814,1827 ----
  			stmt_list = plansource->plan->stmt_list;
  		}
  
+ 		if (!use_single_snap)
+ 			PushActiveSnapshot(GetTransactionSnapshot());
+ 
  		foreach(lc2, stmt_list)
  		{
  			Node	   *stmt = (Node *) lfirst(lc2);
  			bool		canSetTag;
  			DestReceiver *dest;
  
  			_SPI_current->processed = 0;
  			_SPI_current->lastoid = InvalidOid;
***************
*** 1848,1895 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
  
  			/*
  			 * If not read-only mode, advance the command counter before each
! 			 * command.
  			 */
  			if (!read_only)
  				CommandCounterIncrement();
  
  			dest = CreateDestReceiver(canSetTag ? DestSPI : DestNone);
  
- 			if (snapshot == InvalidSnapshot)
- 			{
- 				/*
- 				 * Default read_only behavior is to use the entry-time
- 				 * ActiveSnapshot, if any; if read-write, grab a full new
- 				 * snap.
- 				 */
- 				if (read_only)
- 				{
- 					if (have_active_snap)
- 					{
- 						PushActiveSnapshot(GetActiveSnapshot());
- 						pushed_active_snap = true;
- 					}
- 				}
- 				else
- 				{
- 					PushActiveSnapshot(GetTransactionSnapshot());
- 					pushed_active_snap = true;
- 				}
- 			}
- 			else
- 			{
- 				/*
- 				 * We interpret read_only with a specified snapshot to be
- 				 * exactly that snapshot, but read-write means use the snap
- 				 * with advancing of command ID.
- 				 */
- 				if (read_only)
- 					PushActiveSnapshot(snapshot);
- 				else
- 					PushUpdatedSnapshot(snapshot);
- 				pushed_active_snap = true;
- 			}
- 
  			if (IsA(stmt, PlannedStmt) &&
  				((PlannedStmt *) stmt)->utilityStmt == NULL)
  			{
--- 1862,1877 ----
  
  			/*
  			 * If not read-only mode, advance the command counter before each
! 			 * command and update the snapshot.
  			 */
  			if (!read_only)
+ 			{
  				CommandCounterIncrement();
+ 				UpdateActiveSnapshot();
+ 			}
  
  			dest = CreateDestReceiver(canSetTag ? DestSPI : DestNone);
  
  			if (IsA(stmt, PlannedStmt) &&
  				((PlannedStmt *) stmt)->utilityStmt == NULL)
  			{
***************
*** 1925,1933 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
  				res = SPI_OK_UTILITY;
  			}
  
- 			if (pushed_active_snap)
- 				PopActiveSnapshot();
- 
  			/*
  			 * The last canSetTag query sets the status values returned to the
  			 * caller.	Be careful to free any tuptables not returned, to
--- 1907,1912 ----
***************
*** 1970,1975 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
--- 1949,1958 ----
  
  fail:
  
+ 	/* Pop the last snapshot off the stack if we pushed one */
+ 	if (!read_only || ActiveSnapshotSet())
+ 		PopActiveSnapshot();
+ 
  	/* We no longer need the cached plan refcount, if any */
  	if (cplan)
  		ReleaseCachedPlan(cplan, true);
*** a/src/backend/tcop/postgres.c
--- b/src/backend/tcop/postgres.c
***************
*** 505,552 **** client_read_ended(void)
  	}
  }
  
- 
- /*
-  * Parse a query string and pass it through the rewriter.
-  *
-  * A list of Query nodes is returned, since the string might contain
-  * multiple queries and/or the rewriter might expand one query to several.
-  *
-  * NOTE: this routine is no longer used for processing interactive queries,
-  * but it is still needed for parsing of SQL function bodies.
-  */
- List *
- pg_parse_and_rewrite(const char *query_string,	/* string to execute */
- 					 Oid *paramTypes,	/* parameter types */
- 					 int numParams)		/* number of parameters */
- {
- 	List	   *raw_parsetree_list;
- 	List	   *querytree_list;
- 	ListCell   *list_item;
- 
- 	/*
- 	 * (1) parse the request string into a list of raw parse trees.
- 	 */
- 	raw_parsetree_list = pg_parse_query(query_string);
- 
- 	/*
- 	 * (2) Do parse analysis and rule rewrite.
- 	 */
- 	querytree_list = NIL;
- 	foreach(list_item, raw_parsetree_list)
- 	{
- 		Node	   *parsetree = (Node *) lfirst(list_item);
- 
- 		querytree_list = list_concat(querytree_list,
- 									 pg_analyze_and_rewrite(parsetree,
- 															query_string,
- 															paramTypes,
- 															numParams));
- 	}
- 
- 	return querytree_list;
- }
- 
  /*
   * Do raw parsing (only).
   *
--- 505,510 ----
*** a/src/backend/tcop/pquery.c
--- b/src/backend/tcop/pquery.c
***************
*** 170,180 **** ProcessQuery(PlannedStmt *plan,
  	elog(DEBUG3, "ProcessQuery");
  
  	/*
- 	 * Must always set a snapshot for plannable queries.
- 	 */
- 	PushActiveSnapshot(GetTransactionSnapshot());
- 
- 	/*
  	 * Create the QueryDesc object
  	 */
  	queryDesc = CreateQueryDesc(plan, sourceText,
--- 170,175 ----
***************
*** 234,241 **** ProcessQuery(PlannedStmt *plan,
  	/* Now take care of any queued AFTER triggers */
  	AfterTriggerEndQuery(queryDesc->estate);
  
- 	PopActiveSnapshot();
- 
  	/*
  	 * Now, we close down all the scans and free allocated resources.
  	 */
--- 229,234 ----
***************
*** 1220,1225 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1213,1219 ----
  			   char *completionTag)
  {
  	ListCell   *stmtlist_item;
+ 	bool		pushed_snap = false;
  
  	/*
  	 * If the destination is DestRemoteExecute, change to DestNone.  The
***************
*** 1262,1267 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1256,1270 ----
  			if (log_executor_stats)
  				ResetUsage();
  
+ 			/* if no snapshot is set, grab a new one */
+ 			if (!pushed_snap)
+ 			{
+ 				PushActiveSnapshot(GetTransactionSnapshot());
+ 				pushed_snap = true;
+ 			}
+ 			else
+ 				UpdateActiveSnapshot();
+ 
  			if (pstmt->canSetTag)
  			{
  				/* statement can set tag string */
***************
*** 1291,1301 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1294,1318 ----
  			 *
  			 * These are assumed canSetTag if they're the only stmt in the
  			 * portal.
+ 			 *
+ 			 * NotifyStmt is the only utility statement allowed in a list of
+ 			 * rewritten queries, and it doesn't need a snapshot so we don't
+ 			 * need to worry about it.  However, if the list has only one
+ 			 * statement and it's a utility statement, we are not allowed to
+ 			 * take a snapshot.  See the first comment in PortalRunUtility().
  			 */
  			if (list_length(portal->stmts) == 1)
+ 			{
+ 				Assert(!pushed_snap);
+ 
  				PortalRunUtility(portal, stmt, isTopLevel, dest, completionTag);
+ 			}
  			else
+ 			{
+ 				Assert(IsA(stmt, NotifyStmt));
+ 
  				PortalRunUtility(portal, stmt, isTopLevel, altdest, NULL);
+ 			}
  		}
  
  		/*
***************
*** 1313,1318 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1330,1338 ----
  		MemoryContextDeleteChildren(PortalGetHeapMemory(portal));
  	}
  
+ 	if (pushed_snap)
+ 		PopActiveSnapshot();
+ 
  	/*
  	 * If a command completion tag was supplied, use it.  Otherwise use the
  	 * portal's commandTag as the default completion tag.
*** a/src/backend/utils/time/snapmgr.c
--- b/src/backend/utils/time/snapmgr.c
***************
*** 305,313 **** PushUpdatedSnapshot(Snapshot snapshot)
  	 * elsewhere, so make a new copy to scribble on.
  	 */
  	newsnap = CopySnapshot(snapshot);
- 	newsnap->curcid = GetCurrentCommandId(false);
  
  	PushActiveSnapshot(newsnap);
  }
  
  /*
--- 305,326 ----
  	 * elsewhere, so make a new copy to scribble on.
  	 */
  	newsnap = CopySnapshot(snapshot);
  
  	PushActiveSnapshot(newsnap);
+ 	UpdateActiveSnapshot();
+ }
+ 
+ /*
+  * UpdateActiveSnapshot
+  *
+  * Update the current CID of the active snapshot.
+  */
+ void
+ UpdateActiveSnapshot(void)
+ {
+ 	Assert(ActiveSnapshotSet());
+ 
+ 	ActiveSnapshot->as_snap->curcid = GetCurrentCommandId(false);
  }
  
  /*
*** a/src/include/tcop/tcopprot.h
--- b/src/include/tcop/tcopprot.h
***************
*** 45,52 **** typedef enum
  
  extern int	log_statement;
  
- extern List *pg_parse_and_rewrite(const char *query_string,
- 					 Oid *paramTypes, int numParams);
  extern List *pg_parse_query(const char *query_string);
  extern List *pg_analyze_and_rewrite(Node *parsetree, const char *query_string,
  					   Oid *paramTypes, int numParams);
--- 45,50 ----
*** a/src/include/utils/snapmgr.h
--- b/src/include/utils/snapmgr.h
***************
*** 29,34 **** extern void SnapshotSetCommandId(CommandId curcid);
--- 29,35 ----
  
  extern void PushActiveSnapshot(Snapshot snapshot);
  extern void PushUpdatedSnapshot(Snapshot snapshot);
+ extern void UpdateActiveSnapshot(void);
  extern void PopActiveSnapshot(void);
  extern Snapshot GetActiveSnapshot(void);
  extern bool ActiveSnapshotSet(void);
#20Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Marko Tiikkaja (#19)
1 attachment(s)
Re: Review: Fix snapshot taking inconsistencies

On 2010-10-21 3:32 PM +0200, Marko Tiikkaja wrote:

<patch>

Here's the patch rebased against the master. No code changes since the
last patch I sent.

Regards,
Marko Tiikkaja

Attachments:

snapshot.patchtext/plain; charset=iso-8859-1; name=snapshot.patch; x-mac-creator=0; x-mac-type=0Download
*** a/src/backend/catalog/pg_proc.c
--- b/src/backend/catalog/pg_proc.c
***************
*** 759,765 **** fmgr_sql_validator(PG_FUNCTION_ARGS)
--- 759,767 ----
  	Oid			funcoid = PG_GETARG_OID(0);
  	HeapTuple	tuple;
  	Form_pg_proc proc;
+ 	List	   *raw_parsetree_list;
  	List	   *querytree_list;
+ 	ListCell   *list_item;
  	bool		isnull;
  	Datum		tmp;
  	char	   *prosrc;
***************
*** 832,840 **** fmgr_sql_validator(PG_FUNCTION_ARGS)
  		 */
  		if (!haspolyarg)
  		{
! 			querytree_list = pg_parse_and_rewrite(prosrc,
! 												  proc->proargtypes.values,
! 												  proc->pronargs);
  			(void) check_sql_fn_retval(funcoid, proc->prorettype,
  									   querytree_list,
  									   NULL, NULL);
--- 834,858 ----
  		 */
  		if (!haspolyarg)
  		{
! 			/*
! 			 * Parse and rewrite the queries in the function text.
! 			 *
! 			 * Even though check_sql_fn_retval is only interested in the last
! 			 * query, we analyze all of them here to check for any errors.
! 			 */
! 			raw_parsetree_list = pg_parse_query(prosrc);
! 			
! 			querytree_list = NIL;
! 			foreach(list_item, raw_parsetree_list)
! 			{
! 				Node *parsetree = (Node *) lfirst(list_item);
! 
! 				querytree_list = pg_analyze_and_rewrite(parsetree, prosrc,
! 														proc->proargtypes.values, proc->pronargs);
! 			}
! 
! 			Assert(querytree_list != NIL);
! 
  			(void) check_sql_fn_retval(funcoid, proc->prorettype,
  									   querytree_list,
  									   NULL, NULL);
*** a/src/backend/executor/functions.c
--- b/src/backend/executor/functions.c
***************
*** 84,89 **** typedef struct
--- 84,90 ----
  	bool		returnsSet;		/* true if returning multiple rows */
  	bool		returnsTuple;	/* true if returning whole tuple result */
  	bool		shutdown_reg;	/* true if registered shutdown callback */
+ 	bool		snapshot;		/* true if pushed an active snapshot */
  	bool		readonly_func;	/* true to run in "read only" mode */
  	bool		lazyEval;		/* true if using lazyEval for result query */
  
***************
*** 93,107 **** typedef struct
  
  	JunkFilter *junkFilter;		/* will be NULL if function returns VOID */
  
! 	/* head of linked list of execution_state records */
! 	execution_state *func_state;
  } SQLFunctionCache;
  
  typedef SQLFunctionCache *SQLFunctionCachePtr;
  
  
  /* non-export function prototypes */
! static execution_state *init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK);
  static void init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK);
--- 94,107 ----
  
  	JunkFilter *junkFilter;		/* will be NULL if function returns VOID */
  
! 	List *func_state;
  } SQLFunctionCache;
  
  typedef SQLFunctionCache *SQLFunctionCachePtr;
  
  
  /* non-export function prototypes */
! static List *init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK);
  static void init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK);
***************
*** 123,183 **** static void sqlfunction_destroy(DestReceiver *self);
  
  
  /* Set up the list of per-query execution_state records for a SQL function */
! static execution_state *
  init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK)
  {
! 	execution_state *firstes = NULL;
! 	execution_state *preves = NULL;
  	execution_state *lasttages = NULL;
! 	ListCell   *qtl_item;
  
! 	foreach(qtl_item, queryTree_list)
  	{
! 		Query	   *queryTree = (Query *) lfirst(qtl_item);
! 		Node	   *stmt;
! 		execution_state *newes;
  
! 		Assert(IsA(queryTree, Query));
  
! 		if (queryTree->commandType == CMD_UTILITY)
! 			stmt = queryTree->utilityStmt;
! 		else
! 			stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
  
! 		/* Precheck all commands for validity in a function */
! 		if (IsA(stmt, TransactionStmt))
! 			ereport(ERROR,
! 					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 			/* translator: %s is a SQL statement name */
! 					 errmsg("%s is not allowed in a SQL function",
! 							CreateCommandTag(stmt))));
  
! 		if (fcache->readonly_func && !CommandIsReadOnly(stmt))
! 			ereport(ERROR,
! 					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 			/* translator: %s is a SQL statement name */
! 					 errmsg("%s is not allowed in a non-volatile function",
! 							CreateCommandTag(stmt))));
  
! 		newes = (execution_state *) palloc(sizeof(execution_state));
! 		if (preves)
! 			preves->next = newes;
! 		else
! 			firstes = newes;
  
! 		newes->next = NULL;
! 		newes->status = F_EXEC_START;
! 		newes->setsResult = false;		/* might change below */
! 		newes->lazyEval = false;	/* might change below */
! 		newes->stmt = stmt;
! 		newes->qd = NULL;
  
! 		if (queryTree->canSetTag)
! 			lasttages = newes;
  
! 		preves = newes;
  	}
  
  	/*
--- 123,200 ----
  
  
  /* Set up the list of per-query execution_state records for a SQL function */
! static List *
  init_execution_state(List *queryTree_list,
  					 SQLFunctionCachePtr fcache,
  					 bool lazyEvalOK)
  {
! 	execution_state *firstes;
! 	execution_state *preves;
  	execution_state *lasttages = NULL;
! 	List	   *eslist;
! 	ListCell   *lc1;
! 	ListCell   *lc2;
! 	List	   *qtlist;
! 	Query	   *queryTree;
  
! 
! 	eslist = NIL;
! 
! 	foreach(lc1, queryTree_list)
  	{
! 		qtlist = (List *) lfirst(lc1);
! 		firstes = NULL;
! 		preves = NULL;
  
! 		foreach(lc2, qtlist)
! 		{
! 			Node	   *stmt;
! 			execution_state *newes;
  
! 			queryTree = (Query *) lfirst(lc2);
  
! 			Assert(IsA(queryTree, Query));
  
! 			if (queryTree->commandType == CMD_UTILITY)
! 				stmt = queryTree->utilityStmt;
! 			else
! 				stmt = (Node *) pg_plan_query(queryTree, 0, NULL);
  
! 			/* Precheck all commands for validity in a function */
! 			if (IsA(stmt, TransactionStmt))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 				/* translator: %s is a SQL statement name */
! 						 errmsg("%s is not allowed in a SQL function",
! 								CreateCommandTag(stmt))));
! 
! 			if (fcache->readonly_func && !CommandIsReadOnly(stmt))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 				/* translator: %s is a SQL statement name */
! 						 errmsg("%s is not allowed in a non-volatile function",
! 								CreateCommandTag(stmt))));
! 
! 			newes = (execution_state *) palloc(sizeof(execution_state));
! 			if (preves)
! 				preves->next = newes;
! 			else
! 				firstes = newes;
! 
! 			newes->next = NULL;
! 			newes->status = F_EXEC_START;
! 			newes->setsResult = false;		/* might change below */
! 			newes->lazyEval = false;	/* might change below */
! 			newes->stmt = stmt;
! 			newes->qd = NULL;
  
! 			if (queryTree->canSetTag)
! 				lasttages = newes;
  
! 			preves = newes;
! 		}
  
! 		eslist = lappend(eslist, firstes);
  	}
  
  	/*
***************
*** 210,216 **** init_execution_state(List *queryTree_list,
  		}
  	}
  
! 	return firstes;
  }
  
  /* Initialize the SQLFunctionCache for a SQL function */
--- 227,233 ----
  		}
  	}
  
! 	return eslist;
  }
  
  /* Initialize the SQLFunctionCache for a SQL function */
***************
*** 224,230 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
--- 241,249 ----
  	SQLFunctionCachePtr fcache;
  	Oid		   *argOidVect;
  	int			nargs;
+ 	List	   *raw_parsetree_list;
  	List	   *queryTree_list;
+ 	ListCell   *list_item;
  	Datum		tmp;
  	bool		isNull;
  
***************
*** 319,325 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  	/*
  	 * Parse and rewrite the queries in the function text.
  	 */
! 	queryTree_list = pg_parse_and_rewrite(fcache->src, argOidVect, nargs);
  
  	/*
  	 * Check that the function returns the type it claims to.  Although in
--- 338,356 ----
  	/*
  	 * Parse and rewrite the queries in the function text.
  	 */
! 	raw_parsetree_list = pg_parse_query(fcache->src);
! 
! 	queryTree_list = NIL;
! 	foreach(list_item, raw_parsetree_list)
! 	{
! 		Node *parsetree = (Node *) lfirst(list_item);
! 
! 		queryTree_list = lappend(queryTree_list,
! 								 pg_analyze_and_rewrite(parsetree,
! 									 					fcache->src,
! 														argOidVect,
! 														nargs));
! 	}
  
  	/*
  	 * Check that the function returns the type it claims to.  Although in
***************
*** 342,348 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  	 */
  	fcache->returnsTuple = check_sql_fn_retval(foid,
  											   rettype,
! 											   queryTree_list,
  											   NULL,
  											   &fcache->junkFilter);
  
--- 373,379 ----
  	 */
  	fcache->returnsTuple = check_sql_fn_retval(foid,
  											   rettype,
! 											   llast(queryTree_list),
  											   NULL,
  											   &fcache->junkFilter);
  
***************
*** 374,397 **** init_sql_fcache(FmgrInfo *finfo, bool lazyEvalOK)
  static void
  postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  {
- 	Snapshot	snapshot;
  	DestReceiver *dest;
  
  	Assert(es->qd == NULL);
  
! 	/*
! 	 * In a read-only function, use the surrounding query's snapshot;
! 	 * otherwise take a new snapshot for each query.  The snapshot should
! 	 * include a fresh command ID so that all work to date in this transaction
! 	 * is visible.
! 	 */
! 	if (fcache->readonly_func)
! 		snapshot = GetActiveSnapshot();
! 	else
! 	{
! 		CommandCounterIncrement();
! 		snapshot = GetTransactionSnapshot();
! 	}
  
  	/*
  	 * If this query produces the function result, send its output to the
--- 405,415 ----
  static void
  postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  {
  	DestReceiver *dest;
  
  	Assert(es->qd == NULL);
  
! 	Assert(ActiveSnapshotSet());
  
  	/*
  	 * If this query produces the function result, send its output to the
***************
*** 415,427 **** postquel_start(execution_state *es, SQLFunctionCachePtr fcache)
  	if (IsA(es->stmt, PlannedStmt))
  		es->qd = CreateQueryDesc((PlannedStmt *) es->stmt,
  								 fcache->src,
! 								 snapshot, InvalidSnapshot,
  								 dest,
  								 fcache->paramLI, 0);
  	else
  		es->qd = CreateUtilityQueryDesc(es->stmt,
  										fcache->src,
! 										snapshot,
  										dest,
  										fcache->paramLI);
  
--- 433,446 ----
  	if (IsA(es->stmt, PlannedStmt))
  		es->qd = CreateQueryDesc((PlannedStmt *) es->stmt,
  								 fcache->src,
! 								 GetActiveSnapshot(),
! 								 InvalidSnapshot,
  								 dest,
  								 fcache->paramLI, 0);
  	else
  		es->qd = CreateUtilityQueryDesc(es->stmt,
  										fcache->src,
! 										GetActiveSnapshot(),
  										dest,
  										fcache->paramLI);
  
***************
*** 451,459 **** postquel_getnext(execution_state *es, SQLFunctionCachePtr fcache)
  {
  	bool		result;
  
- 	/* Make our snapshot the active one for any called functions */
- 	PushActiveSnapshot(es->qd->snapshot);
- 
  	if (es->qd->utilitystmt)
  	{
  		/* ProcessUtility needs the PlannedStmt for DECLARE CURSOR */
--- 470,475 ----
***************
*** 481,488 **** postquel_getnext(execution_state *es, SQLFunctionCachePtr fcache)
  		result = (count == 0L || es->qd->estate->es_processed == 0);
  	}
  
- 	PopActiveSnapshot();
- 
  	return result;
  }
  
--- 497,502 ----
***************
*** 496,509 **** postquel_end(execution_state *es)
  	/* Utility commands don't need Executor. */
  	if (es->qd->utilitystmt == NULL)
  	{
- 		/* Make our snapshot the active one for any called functions */
- 		PushActiveSnapshot(es->qd->snapshot);
- 
  		if (es->qd->operation != CMD_SELECT)
  			AfterTriggerEndQuery(es->qd->estate);
  		ExecutorEnd(es->qd);
- 
- 		PopActiveSnapshot();
  	}
  
  	(*es->qd->dest->rDestroy) (es->qd->dest);
--- 510,518 ----
***************
*** 617,622 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 626,633 ----
  	execution_state *es;
  	TupleTableSlot *slot;
  	Datum		result;
+ 	List		*eslist;
+ 	ListCell    *eslc;
  
  	/*
  	 * Switch to context in which the fcache lives.  This ensures that
***************
*** 668,680 **** fmgr_sql(PG_FUNCTION_ARGS)
  		init_sql_fcache(fcinfo->flinfo, lazyEvalOK);
  		fcache = (SQLFunctionCachePtr) fcinfo->flinfo->fn_extra;
  	}
! 	es = fcache->func_state;
  
  	/*
  	 * Convert params to appropriate format if starting a fresh execution. (If
  	 * continuing execution, we can re-use prior params.)
  	 */
! 	if (es && es->status == F_EXEC_START)
  		postquel_sub_params(fcache, fcinfo);
  
  	/*
--- 679,691 ----
  		init_sql_fcache(fcinfo->flinfo, lazyEvalOK);
  		fcache = (SQLFunctionCachePtr) fcinfo->flinfo->fn_extra;
  	}
! 	eslist = fcache->func_state;
  
  	/*
  	 * Convert params to appropriate format if starting a fresh execution. (If
  	 * continuing execution, we can re-use prior params.)
  	 */
! 	if (linitial(eslist) && ((execution_state *) linitial(eslist))->status == F_EXEC_START)
  		postquel_sub_params(fcache, fcinfo);
  
  	/*
***************
*** 687,694 **** fmgr_sql(PG_FUNCTION_ARGS)
  	/*
  	 * Find first unfinished query in function.
  	 */
! 	while (es && es->status == F_EXEC_DONE)
! 		es = es->next;
  
  	/*
  	 * Execute each command in the function one after another until we either
--- 698,715 ----
  	/*
  	 * Find first unfinished query in function.
  	 */
! 
! 	es = NULL; /* keep compiler quiet */
! 	foreach(eslc, eslist)
! 	{
! 		es = (execution_state *) lfirst(eslc);
! 
! 		while (es && es->status == F_EXEC_DONE)
! 			es = es->next;
! 
! 		if (es)
! 			break;
! 	}
  
  	/*
  	 * Execute each command in the function one after another until we either
***************
*** 699,705 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 720,746 ----
  		bool		completed;
  
  		if (es->status == F_EXEC_START)
+ 		{
+ 			if (!fcache->readonly_func)
+ 			{
+ 				/*
+ 				 * In a read-only function, use the surrounding query's snapshot;
+ 				 * otherwise take a new snapshot if we don't have one yet.  The
+ 				 * snapshot should include a fresh command ID so that all work to
+ 				 * date in this transaction is visible.
+ 				 */
+ 				if (!fcache->snapshot)
+ 				{
+ 					CommandCounterIncrement();
+ 					PushActiveSnapshot(GetTransactionSnapshot());
+ 					fcache->snapshot = true;
+ 				}
+ 				else
+ 					UpdateActiveSnapshot();
+ 			}
+ 			
  			postquel_start(es, fcache);
+ 		}
  
  		completed = postquel_getnext(es, fcache);
  
***************
*** 726,731 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 767,791 ----
  		if (es->status != F_EXEC_DONE)
  			break;
  		es = es->next;
+ 
+ 		if (!es)
+ 		{
+ 			eslc = lnext(eslc);
+ 			if (!eslc)
+ 				break;
+ 
+ 			es = (execution_state *) lfirst(eslc);
+ 
+ 			/* make sure we take a new snapshot for this query list */
+ 			if (!fcache->readonly_func)
+ 			{
+ 				Assert(fcache->snapshot);
+ 				PopActiveSnapshot();
+ 				fcache->snapshot = false;
+ 			}
+ 			else
+ 				Assert(!fcache->snapshot);
+ 		}
  	}
  
  	/*
***************
*** 794,799 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 854,864 ----
  											  PointerGetDatum(fcache));
  				fcache->shutdown_reg = false;
  			}
+ 
+ 			/* Release snapshot if we have one */
+ 			if (fcache->snapshot)
+ 				PopActiveSnapshot();
+ 			fcache->snapshot = false;
  		}
  		else
  		{
***************
*** 820,825 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 885,895 ----
  											  PointerGetDatum(fcache));
  				fcache->shutdown_reg = false;
  			}
+ 
+ 			/* Release snapshot if we have one */
+ 			if (fcache->snapshot)
+ 				PopActiveSnapshot();
+ 			fcache->snapshot = false;
  		}
  	}
  	else
***************
*** 850,855 **** fmgr_sql(PG_FUNCTION_ARGS)
--- 920,930 ----
  
  		/* Clear the tuplestore, but keep it for next time */
  		tuplestore_clear(fcache->tstore);
+ 
+ 		/* Release snapshot if we have one */
+ 		if (fcache->snapshot)
+ 			PopActiveSnapshot();
+ 		fcache->snapshot = false;
  	}
  
  	/*
***************
*** 858,868 **** fmgr_sql(PG_FUNCTION_ARGS)
  	 */
  	if (es == NULL)
  	{
! 		es = fcache->func_state;
! 		while (es)
  		{
! 			es->status = F_EXEC_START;
! 			es = es->next;
  		}
  	}
  
--- 933,946 ----
  	 */
  	if (es == NULL)
  	{
! 		foreach(eslc, fcache->func_state)
  		{
! 			es = (execution_state *) lfirst(eslc);
! 			while (es)
! 			{
! 				es->status = F_EXEC_START;
! 				es = es->next;
! 			}
  		}
  	}
  
***************
*** 913,931 **** sql_exec_error_callback(void *arg)
  	{
  		execution_state *es;
  		int			query_num;
  
! 		es = fcache->func_state;
  		query_num = 1;
! 		while (es)
  		{
! 			if (es->qd)
  			{
! 				errcontext("SQL function \"%s\" statement %d",
! 						   fcache->fname, query_num);
! 				break;
  			}
- 			es = es->next;
- 			query_num++;
  		}
  		if (es == NULL)
  		{
--- 991,1014 ----
  	{
  		execution_state *es;
  		int			query_num;
+ 		ListCell   *lc;
  
! 		es = NULL; /* keep compiler quiet */
  		query_num = 1;
! 		foreach(lc, fcache->func_state)
  		{
! 			es = (execution_state *) lfirst(lc);
! 			while (es)
  			{
! 				if (es->qd)
! 				{
! 					errcontext("SQL function \"%s\" statement %d",
! 							   fcache->fname, query_num);
! 					break;
! 				}
! 				es = es->next;
! 				query_num++;
  			}
  		}
  		if (es == NULL)
  		{
***************
*** 956,973 **** static void
  ShutdownSQLFunction(Datum arg)
  {
  	SQLFunctionCachePtr fcache = (SQLFunctionCachePtr) DatumGetPointer(arg);
! 	execution_state *es = fcache->func_state;
  
! 	while (es != NULL)
  	{
! 		/* Shut down anything still running */
! 		if (es->status == F_EXEC_RUN)
! 			postquel_end(es);
! 		/* Reset states to START in case we're called again */
! 		es->status = F_EXEC_START;
! 		es = es->next;
  	}
  
  	/* Release tuplestore if we have one */
  	if (fcache->tstore)
  		tuplestore_end(fcache->tstore);
--- 1039,1067 ----
  ShutdownSQLFunction(Datum arg)
  {
  	SQLFunctionCachePtr fcache = (SQLFunctionCachePtr) DatumGetPointer(arg);
! 	execution_state *es;
! 	ListCell		*lc;
  
! 	foreach(lc, fcache->func_state)
  	{
! 		es = (execution_state *) lfirst(lc);
! 
! 		while (es)
! 		{
! 			/* Shut down anything still running */
! 			if (es->status == F_EXEC_RUN)
! 				postquel_end(es);
! 			/* Reset states to START in case we're called again */
! 			es->status = F_EXEC_START;
! 			es = es->next;
! 		}
  	}
  
+ 	/* Release snapshot if we have one */
+ 	if (fcache->snapshot)
+ 		PopActiveSnapshot();
+ 	fcache->snapshot = false;
+ 
  	/* Release tuplestore if we have one */
  	if (fcache->tstore)
  		tuplestore_end(fcache->tstore);
*** a/src/backend/executor/spi.c
--- b/src/backend/executor/spi.c
***************
*** 1768,1774 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
  	Oid			my_lastoid = InvalidOid;
  	SPITupleTable *my_tuptable = NULL;
  	int			res = 0;
! 	bool		have_active_snap = ActiveSnapshotSet();
  	ErrorContextCallback spierrcontext;
  	CachedPlan *cplan = NULL;
  	ListCell   *lc1;
--- 1768,1774 ----
  	Oid			my_lastoid = InvalidOid;
  	SPITupleTable *my_tuptable = NULL;
  	int			res = 0;
! 	bool		use_single_snap;
  	ErrorContextCallback spierrcontext;
  	CachedPlan *cplan = NULL;
  	ListCell   *lc1;
***************
*** 1781,1786 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
--- 1781,1798 ----
  	spierrcontext.previous = error_context_stack;
  	error_context_stack = &spierrcontext;
  
+ 	if (read_only || snapshot != InvalidSnapshot)
+ 	{
+ 		if (snapshot != InvalidSnapshot)
+ 			PushActiveSnapshot(snapshot);
+ 		else if (read_only && ActiveSnapshotSet())
+ 			PushActiveSnapshot(GetActiveSnapshot());
+ 
+ 		use_single_snap = true;
+ 	}
+ 	else
+ 		use_single_snap = false;
+ 
  	foreach(lc1, plan->plancache_list)
  	{
  		CachedPlanSource *plansource = (CachedPlanSource *) lfirst(lc1);
***************
*** 1802,1813 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
  			stmt_list = plansource->plan->stmt_list;
  		}
  
  		foreach(lc2, stmt_list)
  		{
  			Node	   *stmt = (Node *) lfirst(lc2);
  			bool		canSetTag;
  			DestReceiver *dest;
- 			bool		pushed_active_snap = false;
  
  			_SPI_current->processed = 0;
  			_SPI_current->lastoid = InvalidOid;
--- 1814,1827 ----
  			stmt_list = plansource->plan->stmt_list;
  		}
  
+ 		if (!use_single_snap)
+ 			PushActiveSnapshot(GetTransactionSnapshot());
+ 
  		foreach(lc2, stmt_list)
  		{
  			Node	   *stmt = (Node *) lfirst(lc2);
  			bool		canSetTag;
  			DestReceiver *dest;
  
  			_SPI_current->processed = 0;
  			_SPI_current->lastoid = InvalidOid;
***************
*** 1848,1895 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
  
  			/*
  			 * If not read-only mode, advance the command counter before each
! 			 * command.
  			 */
  			if (!read_only)
  				CommandCounterIncrement();
  
  			dest = CreateDestReceiver(canSetTag ? DestSPI : DestNone);
  
- 			if (snapshot == InvalidSnapshot)
- 			{
- 				/*
- 				 * Default read_only behavior is to use the entry-time
- 				 * ActiveSnapshot, if any; if read-write, grab a full new
- 				 * snap.
- 				 */
- 				if (read_only)
- 				{
- 					if (have_active_snap)
- 					{
- 						PushActiveSnapshot(GetActiveSnapshot());
- 						pushed_active_snap = true;
- 					}
- 				}
- 				else
- 				{
- 					PushActiveSnapshot(GetTransactionSnapshot());
- 					pushed_active_snap = true;
- 				}
- 			}
- 			else
- 			{
- 				/*
- 				 * We interpret read_only with a specified snapshot to be
- 				 * exactly that snapshot, but read-write means use the snap
- 				 * with advancing of command ID.
- 				 */
- 				if (read_only)
- 					PushActiveSnapshot(snapshot);
- 				else
- 					PushUpdatedSnapshot(snapshot);
- 				pushed_active_snap = true;
- 			}
- 
  			if (IsA(stmt, PlannedStmt) &&
  				((PlannedStmt *) stmt)->utilityStmt == NULL)
  			{
--- 1862,1877 ----
  
  			/*
  			 * If not read-only mode, advance the command counter before each
! 			 * command and update the snapshot.
  			 */
  			if (!read_only)
+ 			{
  				CommandCounterIncrement();
+ 				UpdateActiveSnapshot();
+ 			}
  
  			dest = CreateDestReceiver(canSetTag ? DestSPI : DestNone);
  
  			if (IsA(stmt, PlannedStmt) &&
  				((PlannedStmt *) stmt)->utilityStmt == NULL)
  			{
***************
*** 1925,1933 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
  				res = SPI_OK_UTILITY;
  			}
  
- 			if (pushed_active_snap)
- 				PopActiveSnapshot();
- 
  			/*
  			 * The last canSetTag query sets the status values returned to the
  			 * caller.	Be careful to free any tuptables not returned, to
--- 1907,1912 ----
***************
*** 1970,1975 **** _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
--- 1949,1958 ----
  
  fail:
  
+ 	/* Pop the last snapshot off the stack if we pushed one */
+ 	if (!read_only || ActiveSnapshotSet())
+ 		PopActiveSnapshot();
+ 
  	/* We no longer need the cached plan refcount, if any */
  	if (cplan)
  		ReleaseCachedPlan(cplan, true);
*** a/src/backend/tcop/postgres.c
--- b/src/backend/tcop/postgres.c
***************
*** 512,559 **** client_read_ended(void)
  	}
  }
  
- 
- /*
-  * Parse a query string and pass it through the rewriter.
-  *
-  * A list of Query nodes is returned, since the string might contain
-  * multiple queries and/or the rewriter might expand one query to several.
-  *
-  * NOTE: this routine is no longer used for processing interactive queries,
-  * but it is still needed for parsing of SQL function bodies.
-  */
- List *
- pg_parse_and_rewrite(const char *query_string,	/* string to execute */
- 					 Oid *paramTypes,	/* parameter types */
- 					 int numParams)		/* number of parameters */
- {
- 	List	   *raw_parsetree_list;
- 	List	   *querytree_list;
- 	ListCell   *list_item;
- 
- 	/*
- 	 * (1) parse the request string into a list of raw parse trees.
- 	 */
- 	raw_parsetree_list = pg_parse_query(query_string);
- 
- 	/*
- 	 * (2) Do parse analysis and rule rewrite.
- 	 */
- 	querytree_list = NIL;
- 	foreach(list_item, raw_parsetree_list)
- 	{
- 		Node	   *parsetree = (Node *) lfirst(list_item);
- 
- 		querytree_list = list_concat(querytree_list,
- 									 pg_analyze_and_rewrite(parsetree,
- 															query_string,
- 															paramTypes,
- 															numParams));
- 	}
- 
- 	return querytree_list;
- }
- 
  /*
   * Do raw parsing (only).
   *
--- 512,517 ----
*** a/src/backend/tcop/pquery.c
--- b/src/backend/tcop/pquery.c
***************
*** 170,180 **** ProcessQuery(PlannedStmt *plan,
  	elog(DEBUG3, "ProcessQuery");
  
  	/*
- 	 * Must always set a snapshot for plannable queries.
- 	 */
- 	PushActiveSnapshot(GetTransactionSnapshot());
- 
- 	/*
  	 * Create the QueryDesc object
  	 */
  	queryDesc = CreateQueryDesc(plan, sourceText,
--- 170,175 ----
***************
*** 234,241 **** ProcessQuery(PlannedStmt *plan,
  	/* Now take care of any queued AFTER triggers */
  	AfterTriggerEndQuery(queryDesc->estate);
  
- 	PopActiveSnapshot();
- 
  	/*
  	 * Now, we close down all the scans and free allocated resources.
  	 */
--- 229,234 ----
***************
*** 1220,1225 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1213,1219 ----
  			   char *completionTag)
  {
  	ListCell   *stmtlist_item;
+ 	bool		pushed_snap = false;
  
  	/*
  	 * If the destination is DestRemoteExecute, change to DestNone.  The
***************
*** 1262,1267 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1256,1270 ----
  			if (log_executor_stats)
  				ResetUsage();
  
+ 			/* if no snapshot is set, grab a new one */
+ 			if (!pushed_snap)
+ 			{
+ 				PushActiveSnapshot(GetTransactionSnapshot());
+ 				pushed_snap = true;
+ 			}
+ 			else
+ 				UpdateActiveSnapshot();
+ 
  			if (pstmt->canSetTag)
  			{
  				/* statement can set tag string */
***************
*** 1291,1301 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1294,1318 ----
  			 *
  			 * These are assumed canSetTag if they're the only stmt in the
  			 * portal.
+ 			 *
+ 			 * NotifyStmt is the only utility statement allowed in a list of
+ 			 * rewritten queries, and it doesn't need a snapshot so we don't
+ 			 * need to worry about it.  However, if the list has only one
+ 			 * statement and it's a utility statement, we are not allowed to
+ 			 * take a snapshot.  See the first comment in PortalRunUtility().
  			 */
  			if (list_length(portal->stmts) == 1)
+ 			{
+ 				Assert(!pushed_snap);
+ 
  				PortalRunUtility(portal, stmt, isTopLevel, dest, completionTag);
+ 			}
  			else
+ 			{
+ 				Assert(IsA(stmt, NotifyStmt));
+ 
  				PortalRunUtility(portal, stmt, isTopLevel, altdest, NULL);
+ 			}
  		}
  
  		/*
***************
*** 1313,1318 **** PortalRunMulti(Portal portal, bool isTopLevel,
--- 1330,1338 ----
  		MemoryContextDeleteChildren(PortalGetHeapMemory(portal));
  	}
  
+ 	if (pushed_snap)
+ 		PopActiveSnapshot();
+ 
  	/*
  	 * If a command completion tag was supplied, use it.  Otherwise use the
  	 * portal's commandTag as the default completion tag.
*** a/src/backend/utils/time/snapmgr.c
--- b/src/backend/utils/time/snapmgr.c
***************
*** 305,313 **** PushUpdatedSnapshot(Snapshot snapshot)
  	 * elsewhere, so make a new copy to scribble on.
  	 */
  	newsnap = CopySnapshot(snapshot);
- 	newsnap->curcid = GetCurrentCommandId(false);
  
  	PushActiveSnapshot(newsnap);
  }
  
  /*
--- 305,326 ----
  	 * elsewhere, so make a new copy to scribble on.
  	 */
  	newsnap = CopySnapshot(snapshot);
  
  	PushActiveSnapshot(newsnap);
+ 	UpdateActiveSnapshot();
+ }
+ 
+ /*
+  * UpdateActiveSnapshot
+  *
+  * Update the current CID of the active snapshot.
+  */
+ void
+ UpdateActiveSnapshot(void)
+ {
+ 	Assert(ActiveSnapshotSet());
+ 
+ 	ActiveSnapshot->as_snap->curcid = GetCurrentCommandId(false);
  }
  
  /*
*** a/src/include/tcop/tcopprot.h
--- b/src/include/tcop/tcopprot.h
***************
*** 45,52 **** typedef enum
  
  extern int	log_statement;
  
- extern List *pg_parse_and_rewrite(const char *query_string,
- 					 Oid *paramTypes, int numParams);
  extern List *pg_parse_query(const char *query_string);
  extern List *pg_analyze_and_rewrite(Node *parsetree, const char *query_string,
  					   Oid *paramTypes, int numParams);
--- 45,50 ----
*** a/src/include/utils/snapmgr.h
--- b/src/include/utils/snapmgr.h
***************
*** 29,34 **** extern void SnapshotSetCommandId(CommandId curcid);
--- 29,35 ----
  
  extern void PushActiveSnapshot(Snapshot snapshot);
  extern void PushUpdatedSnapshot(Snapshot snapshot);
+ extern void UpdateActiveSnapshot(void);
  extern void PopActiveSnapshot(void);
  extern Snapshot GetActiveSnapshot(void);
  extern bool ActiveSnapshotSet(void);
#21Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#18)
Re: Review: Fix snapshot taking inconsistencies

On Wed, Oct 20, 2010 at 6:15 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Alvaro Herrera <alvherre@commandprompt.com> writes:

It strikes me that if we really want to restructure things to divide
client interaction from other query-processing routines, we should
create another file, say src/backend/tcop/queries.c; this would have
stuff like pg_plan_query, pg_plan_queries, pg_rewrite_query, and the
other things that the patch was evicting from postgres.c (plus, I
imagine, a bunch of other stuff that I may be missing).  In fact, if we
go down this route, there would be no point in removing
pg_parse_and_rewrite; we would just move it to this new module.

Yeah, possibly that would be a good idea.

To my mind, the first thing that has to be resolved before messing
around in this area is whether or not we want the logging/statistics
behavior of these functions to apply when they are used in contexts
other than interactive queries.  Personally I would vote no, mainly
because I don't think that behavior is very sensible in nested
execution.  If that is the decision, then probably these functions
should stay where they are and as they are, and we just deprecate
outside use of them.  I'm not sure whether there's enough left after
deleting the logging/statistics behavior to justify making exported
versions, as opposed to just having the callers call the next-layer-down
functionality directly.

It looks to me like the log_planner_stats stuff will blow up in nested
execution. So there's certainly no point in doing that.

The debug_print_plan stuff *might* be useful in nested execution...
although I'm not convinced. Not too many people are probably going to
use this at all, since the output is not human-readable.

I'm not real sure about the dtrace probes. If they are useful in
nested execution, we could move them down a bit (e.g. put
TRACE_POSTGRESQL_QUERY_PLAN_START/END into planner()).

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#22Alvaro Herrera
alvherre@commandprompt.com
In reply to: Marko Tiikkaja (#20)
Re: Review: Fix snapshot taking inconsistencies

Excerpts from Marko Tiikkaja's message of sáb ene 15 17:30:14 -0300 2011:

On 2010-10-21 3:32 PM +0200, Marko Tiikkaja wrote:

<patch>

Here's the patch rebased against the master. No code changes since the
last patch I sent.

Having a look at this.

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

#23Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#22)
Re: Review: Fix snapshot taking inconsistencies

Alvaro Herrera <alvherre@commandprompt.com> writes:

Excerpts from Marko Tiikkaja's message of sáb ene 15 17:30:14 -0300 2011:

On 2010-10-21 3:32 PM +0200, Marko Tiikkaja wrote:

<patch>

Here's the patch rebased against the master. No code changes since the
last patch I sent.

Having a look at this.

My recollection is that this was pretty tightly coupled to the wCTE
patch. I had been intending to review them together, and have just
now come up for air enough to start doing that. Do you really want
to review this one separately?

regards, tom lane

#24Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Tom Lane (#23)
Re: Review: Fix snapshot taking inconsistencies

On 2011-02-24 12:39 AM, Tom Lane wrote:

My recollection is that this was pretty tightly coupled to the wCTE
patch.

It was, but isn't anymore. Now it's just a bugfix.

Regards,
Marko Tiikkaja

#25Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#24)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2011-02-24 12:39 AM, Tom Lane wrote:

My recollection is that this was pretty tightly coupled to the wCTE
patch.

It was, but isn't anymore. Now it's just a bugfix.

The connection is the question of where to do CommandCounterIncrement
between successive DML WITH operations in a single command. Right
offhand, I don't see any CommandCounterIncrement in the wCTE patch,
so I'm sort of wondering whether the case works at all...

regards, tom lane

#26Alvaro Herrera
alvherre@commandprompt.com
In reply to: Tom Lane (#23)
Re: Review: Fix snapshot taking inconsistencies

Excerpts from Tom Lane's message of mié feb 23 19:39:23 -0300 2011:

Alvaro Herrera <alvherre@commandprompt.com> writes:

Excerpts from Marko Tiikkaja's message of sáb ene 15 17:30:14 -0300 2011:

On 2010-10-21 3:32 PM +0200, Marko Tiikkaja wrote:

<patch>

Here's the patch rebased against the master. No code changes since the
last patch I sent.

Having a look at this.

My recollection is that this was pretty tightly coupled to the wCTE
patch. I had been intending to review them together, and have just
now come up for air enough to start doing that. Do you really want
to review this one separately?

Dunno. If you're gonna pick it up I guess my time is best spent
elsewhere. There was some restructuring in code in postgres.c to be
done near this patch, which wasn't attacked at all by Marko AFAICS.
Maybe I should be looking at that instead.

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

#27Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Tom Lane (#25)
Re: Review: Fix snapshot taking inconsistencies

On 2011-02-24 2:31 AM, Tom Lane wrote:

Marko Tiikkaja<marko.tiikkaja@cs.helsinki.fi> writes:

On 2011-02-24 12:39 AM, Tom Lane wrote:

My recollection is that this was pretty tightly coupled to the wCTE
patch.

It was, but isn't anymore. Now it's just a bugfix.

The connection is the question of where to do CommandCounterIncrement
between successive DML WITH operations in a single command.

.. what? We decided *not* to do any CommandCounterIncrements between
DML WITHs.

Regards,
Marko Tiikkaja

#28Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#26)
Re: Review: Fix snapshot taking inconsistencies

Alvaro Herrera <alvherre@commandprompt.com> writes:

Excerpts from Tom Lane's message of mié feb 23 19:39:23 -0300 2011:

My recollection is that this was pretty tightly coupled to the wCTE
patch. I had been intending to review them together, and have just
now come up for air enough to start doing that. Do you really want
to review this one separately?

Dunno. If you're gonna pick it up I guess my time is best spent
elsewhere. There was some restructuring in code in postgres.c to be
done near this patch, which wasn't attacked at all by Marko AFAICS.
Maybe I should be looking at that instead.

Well, Marko claims they're independent, so if you feel it fits into
what you're doing you're welcome to it. But I was planning to deal
with Marko's two patches as soon as the FDW dust settled, and it
seems to be settled.

regards, tom lane

#29Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Alvaro Herrera (#26)
Re: Review: Fix snapshot taking inconsistencies

On 2011-02-24 2:35 AM, Alvaro Herrera wrote:

There was some restructuring in code in postgres.c to be
done near this patch, which wasn't attacked at all by Marko AFAICS.
Maybe I should be looking at that instead.

I don't feel at all comfortable doing the restructuring you guys have
been talking about.

Regards,
Marko Tiikkaja

#30Alvaro Herrera
alvherre@commandprompt.com
In reply to: Alvaro Herrera (#26)
Re: Review: Fix snapshot taking inconsistencies

Excerpts from Alvaro Herrera's message of mié feb 23 21:35:13 -0300 2011:

Excerpts from Tom Lane's message of mié feb 23 19:39:23 -0300 2011:

My recollection is that this was pretty tightly coupled to the wCTE
patch. I had been intending to review them together, and have just
now come up for air enough to start doing that. Do you really want
to review this one separately?

Dunno. If you're gonna pick it up I guess my time is best spent
elsewhere. There was some restructuring in code in postgres.c to be
done near this patch, which wasn't attacked at all by Marko AFAICS.
Maybe I should be looking at that instead.

... but then, if I did that, I could create merge conflicts for you, so
please have at it. I think pure beautification code cleanups can way
till 9.2 starts.

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

#31Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#27)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2011-02-24 2:31 AM, Tom Lane wrote:

The connection is the question of where to do CommandCounterIncrement
between successive DML WITH operations in a single command.

.. what? We decided *not* to do any CommandCounterIncrements between
DML WITHs.

Oh, did we decide to do it that way? OK with me, but the submitted docs
are woefully inadequate on the point. This behavior is going to have to
be explained extremely clearly (and even so, I bet we'll get bug reports
about it :-().

regards, tom lane

#32Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Tom Lane (#31)
Re: Review: Fix snapshot taking inconsistencies

On 2011-02-24 5:21 PM, Tom Lane wrote:

Oh, did we decide to do it that way? OK with me, but the submitted docs
are woefully inadequate on the point. This behavior is going to have to
be explained extremely clearly (and even so, I bet we'll get bug reports
about it :-().

I'm ready to put more effort into the documentation if the patch is
going in, but I really don't want to waste my time just to hear that the
patch is not going to be in 9.1. Does this sound acceptable?

Regards,
Marko Tiikkaja

#33Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#32)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2011-02-24 5:21 PM, Tom Lane wrote:

Oh, did we decide to do it that way? OK with me, but the submitted docs
are woefully inadequate on the point. This behavior is going to have to
be explained extremely clearly (and even so, I bet we'll get bug reports
about it :-().

I'm ready to put more effort into the documentation if the patch is
going in, but I really don't want to waste my time just to hear that the
patch is not going to be in 9.1. Does this sound acceptable?

I've found some things I don't like about it, but the only part that
seems far short of being committable is the documentation.

regards, tom lane

#34Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#33)
Re: Review: Fix snapshot taking inconsistencies

On Thu, Feb 24, 2011 at 11:02 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2011-02-24 5:21 PM, Tom Lane wrote:

Oh, did we decide to do it that way?  OK with me, but the submitted docs
are woefully inadequate on the point.  This behavior is going to have to
be explained extremely clearly (and even so, I bet we'll get bug reports
about it :-().

I'm ready to put more effort into the documentation if the patch is
going in, but I really don't want to waste my time just to hear that the
patch is not going to be in 9.1.  Does this sound acceptable?

I've found some things I don't like about it, but the only part that
seems far short of being committable is the documentation.

Tom/Alvaro, have the two of you hammered out who is going to finish
this one off? I *believe* Alvaro told me on IM that he was leaving
this one for Tom.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#35Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#34)
Re: Review: Fix snapshot taking inconsistencies

Robert Haas <robertmhaas@gmail.com> writes:

Tom/Alvaro, have the two of you hammered out who is going to finish
this one off? I *believe* Alvaro told me on IM that he was leaving
this one for Tom.

Last I heard, the ball was in my court. I'll try to get it done over
the weekend.

regards, tom lane

#36Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#20)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

[ latest version of snapshot-taking patch ]

I started to look at this, and find myself fairly confused as to what
the purpose is anymore. Reviewing the thread, there has been a lot of
discussion of refactoring the API of pg_parse_and_rewrite and related
functions exported by postgres.c; but the current patch seems to have
abandoned that goal (except for removing pg_parse_and_rewrite itself,
which doesn't seem to me to have a lot of point except as part of a
more general refactoring). With respect to the issue of changing
snapshot timing, most of the discussion around that seemed to start
from assumptions about the behavior of wCTEs that we've now abandoned.
And there was some discussion about rule behavior too, but it's not
clear to me whether this patch intends to change that or not. The
lack of any documentation change doesn't help here.

So: exactly what is the intended behavioral change as of now, and what
is the argument supporting that change?

regards, tom lane

#37Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#36)
Re: Review: Fix snapshot taking inconsistencies

On Mon, Feb 28, 2011 at 1:22 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

[ latest version of snapshot-taking patch ]

I started to look at this, and find myself fairly confused as to what
the purpose is anymore.  Reviewing the thread, there has been a lot of
discussion of refactoring the API of pg_parse_and_rewrite and related
functions exported by postgres.c; but the current patch seems to have
abandoned that goal (except for removing pg_parse_and_rewrite itself,
which doesn't seem to me to have a lot of point except as part of a
more general refactoring).  With respect to the issue of changing
snapshot timing, most of the discussion around that seemed to start
from assumptions about the behavior of wCTEs that we've now abandoned.
And there was some discussion about rule behavior too, but it's not
clear to me whether this patch intends to change that or not.  The
lack of any documentation change doesn't help here.

So: exactly what is the intended behavioral change as of now, and what
is the argument supporting that change?

IIUC, this is the result of countless rounds of communal bikeshedding around:

http://archives.postgresql.org/pgsql-hackers/2010-07/msg01256.php

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#38Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#37)
Re: Review: Fix snapshot taking inconsistencies

Robert Haas <robertmhaas@gmail.com> writes:

On Mon, Feb 28, 2011 at 1:22 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

So: exactly what is the intended behavioral change as of now, and what
is the argument supporting that change?

IIUC, this is the result of countless rounds of communal bikeshedding around:

Quite :-(. But I'm not sure where the merry-go-round stopped.

http://archives.postgresql.org/pgsql-hackers/2010-07/msg01256.php

Please notice that the very terms of discussion in that message depend
on a view of wCTEs that has got nothing to do with what was applied.
I'm afraid that the goals of this patch might be similarly obsolete.
I definitely don't want to apply the patch in a hurry just because
we're down to the end of the commitfest.

regards, tom lane

#39Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#38)
Re: Review: Fix snapshot taking inconsistencies

On Mon, Feb 28, 2011 at 1:45 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Mon, Feb 28, 2011 at 1:22 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

So: exactly what is the intended behavioral change as of now, and what
is the argument supporting that change?

IIUC, this is the result of countless rounds of communal bikeshedding around:

Quite :-(.  But I'm not sure where the merry-go-round stopped.

http://archives.postgresql.org/pgsql-hackers/2010-07/msg01256.php

Please notice that the very terms of discussion in that message depend
on a view of wCTEs that has got nothing to do with what was applied.
I'm afraid that the goals of this patch might be similarly obsolete.

No, I don't think so. IIUC, the problem is that EXPLAIN ANALYZE runs
the rewrite products with different snapshot handling than the regular
execution path. So in theory you could turn on auto_explain and have
the semantics of your queries change. That would be Bad.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#40Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Tom Lane (#36)
Re: Review: Fix snapshot taking inconsistencies

On 2011-02-28 8:22 PM, Tom Lane wrote:

Marko Tiikkaja<marko.tiikkaja@cs.helsinki.fi> writes:

[ latest version of snapshot-taking patch ]

I started to look at this, and find myself fairly confused as to what
the purpose is anymore. Reviewing the thread, there has been a lot of
discussion of refactoring the API of pg_parse_and_rewrite and related
functions exported by postgres.c; but the current patch seems to have
abandoned that goal (except for removing pg_parse_and_rewrite itself,
which doesn't seem to me to have a lot of point except as part of a
more general refactoring). With respect to the issue of changing
snapshot timing, most of the discussion around that seemed to start
from assumptions about the behavior of wCTEs that we've now abandoned.
And there was some discussion about rule behavior too, but it's not
clear to me whether this patch intends to change that or not. The
lack of any documentation change doesn't help here.

So: exactly what is the intended behavioral change as of now, and what
is the argument supporting that change?

The only intended change is what I was wondering in the original post:
snapshot handling between normal execution and EXPLAIN ANALYZE when a
query expands to multiple trees because of rewrite rules. Like I said
earlier, this is just a bugfix now that wCTEs don't need it anymore.

Rcgards,
Marko Tiikkaja

#41Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#39)
Re: Review: Fix snapshot taking inconsistencies

Robert Haas <robertmhaas@gmail.com> writes:

On Mon, Feb 28, 2011 at 1:45 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I'm afraid that the goals of this patch might be similarly obsolete.

No, I don't think so. IIUC, the problem is that EXPLAIN ANALYZE runs
the rewrite products with different snapshot handling than the regular
execution path.

Possibly, but it's not clear to me that this patch fixes that.
As I said, it's no longer obvious what the patch means to do, and I'd
like a clear statement of that.

So in theory you could turn on auto_explain and have
the semantics of your queries change. That would be Bad.

That's just FUD. auto_explain doesn't run EXPLAIN ANALYZE.

regards, tom lane

#42Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#40)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2011-02-28 8:22 PM, Tom Lane wrote:

So: exactly what is the intended behavioral change as of now, and what
is the argument supporting that change?

The only intended change is what I was wondering in the original post:
snapshot handling between normal execution and EXPLAIN ANALYZE when a
query expands to multiple trees because of rewrite rules. Like I said
earlier, this is just a bugfix now that wCTEs don't need it anymore.

OK, and which behavior is getting changed, to what? I am not interested
in trying to reverse-engineer a specification from the patch.

regards, tom lane

#43Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#41)
Re: Review: Fix snapshot taking inconsistencies

On Mon, Feb 28, 2011 at 2:01 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Mon, Feb 28, 2011 at 1:45 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I'm afraid that the goals of this patch might be similarly obsolete.

No, I don't think so.  IIUC, the problem is that EXPLAIN ANALYZE runs
the rewrite products with different snapshot handling than the regular
execution path.

Possibly, but it's not clear to me that this patch fixes that.
As I said, it's no longer obvious what the patch means to do, and I'd
like a clear statement of that.

Fair enough. I assume Marko will provide that shortly. I believe the
consensus was to make the regular case behave like EXPLAIN ANALYZE
rather than the other way around...

So in theory you could turn on auto_explain and have
the semantics of your queries change.  That would be Bad.

That's just FUD.  auto_explain doesn't run EXPLAIN ANALYZE.

Oh, woops. I stand corrected. But I guess the query might behave
differently with and without EXPLAIN ANALYZE?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#44Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Tom Lane (#42)
Re: Review: Fix snapshot taking inconsistencies

On 2011-02-28 9:03 PM, Tom Lane wrote:

Marko Tiikkaja<marko.tiikkaja@cs.helsinki.fi> writes:

On 2011-02-28 8:22 PM, Tom Lane wrote:

So: exactly what is the intended behavioral change as of now, and what
is the argument supporting that change?

The only intended change is what I was wondering in the original post:
snapshot handling between normal execution and EXPLAIN ANALYZE when a
query expands to multiple trees because of rewrite rules. Like I said
earlier, this is just a bugfix now that wCTEs don't need it anymore.

OK, and which behavior is getting changed, to what? I am not interested
in trying to reverse-engineer a specification from the patch.

My recollection is (and the archives seem to agree) that normal
execution and SQL functions were changed to only advance the CID instead
of taking a new snapshot. EXPLAIN ANALYZE and SPI (not exactly sure
about this one) did that already so they were just changed to use the
new API.

Regards,
Marko Tiikkaja

#45Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#44)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2011-02-28 9:03 PM, Tom Lane wrote:

OK, and which behavior is getting changed, to what? I am not interested
in trying to reverse-engineer a specification from the patch.

My recollection is (and the archives seem to agree) that normal
execution and SQL functions were changed to only advance the CID instead
of taking a new snapshot. EXPLAIN ANALYZE and SPI (not exactly sure
about this one) did that already so they were just changed to use the
new API.

OK, so the intent is that in all cases, we just advance CID and don't
take a new snapshot between queries that were generated (by rule
expansion) from a single original parsetree? But we still take a new
snap between original parsetrees? Works for me.

regards, tom lane

#46Marko Tiikkaja
marko.tiikkaja@cs.helsinki.fi
In reply to: Tom Lane (#45)
Re: Review: Fix snapshot taking inconsistencies

On 2011-02-28 9:36 PM, Tom Lane wrote:

Marko Tiikkaja<marko.tiikkaja@cs.helsinki.fi> writes:

On 2011-02-28 9:03 PM, Tom Lane wrote:

OK, and which behavior is getting changed, to what? I am not interested
in trying to reverse-engineer a specification from the patch.

My recollection is (and the archives seem to agree) that normal
execution and SQL functions were changed to only advance the CID instead
of taking a new snapshot. EXPLAIN ANALYZE and SPI (not exactly sure
about this one) did that already so they were just changed to use the
new API.

OK, so the intent is that in all cases, we just advance CID and don't
take a new snapshot between queries that were generated (by rule
expansion) from a single original parsetree? But we still take a new
snap between original parsetrees? Works for me.

Exactly.

Regards,
Marko Tiikkaja

#47Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Tiikkaja (#46)
Re: Review: Fix snapshot taking inconsistencies

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> writes:

On 2011-02-28 9:36 PM, Tom Lane wrote:

OK, so the intent is that in all cases, we just advance CID and don't
take a new snapshot between queries that were generated (by rule
expansion) from a single original parsetree? But we still take a new
snap between original parsetrees? Works for me.

Exactly.

OK, applied with corrections (I didn't think either the spi.c or
functions.c changes were quite right).

regards, tom lane