EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
Started by Bharath Rupireddyabout 5 years ago27 messages
Hi,
Currently, $subject is not allowed. We do plan the mat view query
before every refresh. I propose to show the explain/explain analyze of
the select part of the mat view in case of Refresh Mat View(RMV). It
will be useful for the user to know what exactly is being planned and
executed as part of RMV. Please note that we already have
explain/explain analyze CTAS/Create Mat View(CMV), where we show the
explain/explain analyze of the select part. This proposal will do the
same thing.
The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.
Thoughts? If okay, I will post a patch later.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Tue, Dec 22, 2020 at 7:01 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
Currently, $subject is not allowed. We do plan the mat view query
before every refresh. I propose to show the explain/explain analyze of
the select part of the mat view in case of Refresh Mat View(RMV). It
will be useful for the user to know what exactly is being planned and
executed as part of RMV. Please note that we already have
explain/explain analyze CTAS/Create Mat View(CMV), where we show the
explain/explain analyze of the select part. This proposal will do the
same thing.The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.Thoughts? If okay, I will post a patch later.
Attaching below patches:
0001 - Rearrange Refresh Mat View Code - Currently, the function
ExecRefreshMatView in matview.c is having many lines of code which is
not at all good from readability and maintainability perspectives.
This patch adds a few functions and moves the code from
ExecRefreshMatView to them making the code look better.
0002 - EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW support and tests.
If this proposal is useful, I have few open points - 1) In the patch I
have added a new mat view info parameter to ExplainOneQuery(), do we
also need to add it to ExplainOneQuery_hook_type? 2) Do we document
(under respective command pages or somewhere else) that we allow
explain/explain analyze for a command?
Thoughts?
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v1-0001-Rearrange-Refresh-Mat-View-Code.patchapplication/x-patch; name=v1-0001-Rearrange-Refresh-Mat-View-Code.patchDownload
From e0422b72acbaed27182ad3816cdc921a0f962fe2 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Fri, 25 Dec 2020 14:03:17 +0530
Subject: [PATCH v1] Rearrange Refresh Mat View Code
Currently, the function ExecRefreshMatView in matview.c is having
many lines of code which is not at all good from readability and
maintainability perspectives. This patch adds few functions and
moves the code from ExecRefreshMatView to them making the code
look better.
---
src/backend/commands/matview.c | 452 ++++++++++++++++++++-------------
1 file changed, 273 insertions(+), 179 deletions(-)
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index cfc63915f3..40cb436d16 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -64,7 +64,7 @@ static void transientrel_startup(DestReceiver *self, int operation, TupleDesc ty
static bool transientrel_receive(TupleTableSlot *slot, DestReceiver *self);
static void transientrel_shutdown(DestReceiver *self);
static void transientrel_destroy(DestReceiver *self);
-static uint64 refresh_matview_datafill(DestReceiver *dest, Query *query,
+static uint64 refresh_matview_datafill(Oid OIDNewHeap, Query *query,
const char *queryString);
static char *make_temptable_name_n(char *tempname, int n);
static void refresh_by_match_merge(Oid matviewOid, Oid tempOid, Oid relowner,
@@ -73,6 +73,16 @@ static void refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersist
static bool is_usable_unique_index(Relation indexRel);
static void OpenMatViewIncrementalMaintenance(void);
static void CloseMatViewIncrementalMaintenance(void);
+static Query *get_matview_query(RefreshMatViewStmt *stmt, Relation *rel,
+ Oid *objectId);
+static Query *rewrite_refresh_matview_query(Query *dataQuery);
+static Oid get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, char *relpersistence);
+static void match_matview_with_new_data(RefreshMatViewStmt *stmt,
+ Relation matviewRel, Oid matviewOid,
+ Oid OIDNewHeap, Oid relowner,
+ int save_sec_context,
+ char relpersistence, uint64 processed);
/*
* SetMatViewPopulatedState
@@ -140,127 +150,20 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
{
Oid matviewOid;
Relation matviewRel;
- RewriteRule *rule;
- List *actions;
Query *dataQuery;
- Oid tableSpace;
- Oid relowner;
Oid OIDNewHeap;
- DestReceiver *dest;
uint64 processed = 0;
- bool concurrent;
- LOCKMODE lockmode;
+ Oid relowner;
char relpersistence;
Oid save_userid;
int save_sec_context;
int save_nestlevel;
ObjectAddress address;
- /* Determine strength of lock needed. */
- concurrent = stmt->concurrent;
- lockmode = concurrent ? ExclusiveLock : AccessExclusiveLock;
-
- /*
- * Get a lock until end of transaction.
- */
- matviewOid = RangeVarGetRelidExtended(stmt->relation,
- lockmode, 0,
- RangeVarCallbackOwnsTable, NULL);
- matviewRel = table_open(matviewOid, NoLock);
-
- /* Make sure it is a materialized view. */
- if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("\"%s\" is not a materialized view",
- RelationGetRelationName(matviewRel))));
-
- /* Check that CONCURRENTLY is not specified if not populated. */
- if (concurrent && !RelationIsPopulated(matviewRel))
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
-
- /* Check that conflicting options have not been specified. */
- if (concurrent && stmt->skipData)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
-
- /*
- * Check that everything is correct for a refresh. Problems at this point
- * are internal errors, so elog is sufficient.
- */
- if (matviewRel->rd_rel->relhasrules == false ||
- matviewRel->rd_rules->numLocks < 1)
- elog(ERROR,
- "materialized view \"%s\" is missing rewrite information",
- RelationGetRelationName(matviewRel));
-
- if (matviewRel->rd_rules->numLocks > 1)
- elog(ERROR,
- "materialized view \"%s\" has too many rules",
- RelationGetRelationName(matviewRel));
-
- rule = matviewRel->rd_rules->rules[0];
- if (rule->event != CMD_SELECT || !(rule->isInstead))
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
- RelationGetRelationName(matviewRel));
-
- actions = rule->actions;
- if (list_length(actions) != 1)
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a single action",
- RelationGetRelationName(matviewRel));
-
- /*
- * Check that there is a unique index with no WHERE clause on one or more
- * columns of the materialized view if CONCURRENTLY is specified.
- */
- if (concurrent)
- {
- List *indexoidlist = RelationGetIndexList(matviewRel);
- ListCell *indexoidscan;
- bool hasUniqueIndex = false;
-
- foreach(indexoidscan, indexoidlist)
- {
- Oid indexoid = lfirst_oid(indexoidscan);
- Relation indexRel;
-
- indexRel = index_open(indexoid, AccessShareLock);
- hasUniqueIndex = is_usable_unique_index(indexRel);
- index_close(indexRel, AccessShareLock);
- if (hasUniqueIndex)
- break;
- }
-
- list_free(indexoidlist);
-
- if (!hasUniqueIndex)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot refresh materialized view \"%s\" concurrently",
- quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
- RelationGetRelationName(matviewRel))),
- errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
- }
-
- /*
- * The stored query was rewritten at the time of the MV definition, but
- * has not been scribbled on by the planner.
- */
- dataQuery = linitial_node(Query, actions);
+ /* Get the data generating query. */
+ dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);
- /*
- * Check for active uses of the relation in the current transaction, such
- * as open scans.
- *
- * NB: We count on this to protect us against problems with refreshing the
- * data using TABLE_INSERT_FROZEN.
- */
- CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+ relowner = matviewRel->rd_rel->relowner;
/*
* Tentatively mark the matview as populated or not (this will roll back
@@ -281,27 +184,8 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- /* Concurrent refresh builds new data in temp tablespace, and does diff. */
- if (concurrent)
- {
- tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
- relpersistence = RELPERSISTENCE_TEMP;
- }
- else
- {
- tableSpace = matviewRel->rd_rel->reltablespace;
- relpersistence = matviewRel->rd_rel->relpersistence;
- }
-
- /*
- * Create the transient table that will receive the regenerated data. Lock
- * it against access by any other process until commit (by which time it
- * will be gone).
- */
- OIDNewHeap = make_new_heap(matviewOid, tableSpace, relpersistence,
- ExclusiveLock);
- LockRelationOid(OIDNewHeap, AccessExclusiveLock);
- dest = CreateTransientRelDestReceiver(OIDNewHeap);
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -311,40 +195,16 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/* Generate the data, if wanted. */
if (!stmt->skipData)
- processed = refresh_matview_datafill(dest, dataQuery, queryString);
-
- /* Make the matview match the newly generated data. */
- if (concurrent)
{
- int old_depth = matview_maintenance_depth;
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
- PG_TRY();
- {
- refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
- save_sec_context);
- }
- PG_CATCH();
- {
- matview_maintenance_depth = old_depth;
- PG_RE_THROW();
- }
- PG_END_TRY();
- Assert(matview_maintenance_depth == old_depth);
+ processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
+ queryString);
}
- else
- {
- refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
- /*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
- */
- pgstat_count_truncate(matviewRel);
- if (!stmt->skipData)
- pgstat_count_heap_insert(matviewRel, processed);
- }
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
@@ -373,30 +233,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/*
* refresh_matview_datafill
*
- * Execute the given query, sending result rows to "dest" (which will
- * insert them into the target matview).
+ * Create dest receiver and execute the given query, sending result rows to the
+ * dest receiver which will insert them into the target materialized view.
*
* Returns number of rows inserted.
*/
static uint64
-refresh_matview_datafill(DestReceiver *dest, Query *query,
- const char *queryString)
+refresh_matview_datafill(Oid OIDNewHeap, Query *query, const char *queryString)
{
- List *rewritten;
PlannedStmt *plan;
QueryDesc *queryDesc;
- Query *copied_query;
uint64 processed;
-
- /* Lock and rewrite, using a copy to preserve the original query. */
- copied_query = copyObject(query);
- AcquireRewriteLocks(copied_query, true, false);
- rewritten = QueryRewrite(copied_query);
-
- /* SELECT should never rewrite to more or less than one SELECT query */
- if (list_length(rewritten) != 1)
- elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
- query = (Query *) linitial(rewritten);
+ DestReceiver *dest;
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
@@ -413,6 +261,8 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
PushCopiedSnapshot(GetActiveSnapshot());
UpdateActiveSnapshotCommandId();
+ dest = CreateTransientRelDestReceiver(OIDNewHeap);
+
/* Create a QueryDesc, redirecting output to our tuple receiver */
queryDesc = CreateQueryDesc(plan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
@@ -933,3 +783,247 @@ CloseMatViewIncrementalMaintenance(void)
matview_maintenance_depth--;
Assert(matview_maintenance_depth >= 0);
}
+
+/*
+ * get_matview_query
+ *
+ * Open the refresh materialized view relation, perform sanity checks and also
+ * get the associated data generating query from it.
+ *
+ * Note that the refresh materialized view relation is opened here, it has to
+ * be closed in the caller.
+ */
+static Query *
+get_matview_query(RefreshMatViewStmt *stmt, Relation *rel, Oid *objectId)
+{
+ Oid matviewOid;
+ Relation matviewRel;
+ RewriteRule *rule;
+ List *actions;
+ Query *dataQuery;
+ LOCKMODE lockmode;
+
+ /* Determine strength of lock needed. */
+ lockmode = stmt->concurrent ? ExclusiveLock : AccessExclusiveLock;
+
+ /* Get a lock until end of transaction. */
+ matviewOid = RangeVarGetRelidExtended(stmt->relation, lockmode, 0,
+ RangeVarCallbackOwnsTable, NULL);
+ matviewRel = table_open(matviewOid, NoLock);
+
+ /* Make sure it is a materialized view. */
+ if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("\"%s\" is not a materialized view",
+ RelationGetRelationName(matviewRel))));
+
+ /* Check that CONCURRENTLY is not specified if not populated. */
+ if (stmt->concurrent && !RelationIsPopulated(matviewRel))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
+
+ /* Check that conflicting options have not been specified. */
+ if (stmt->concurrent && stmt->skipData)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
+
+ /*
+ * Check that everything is correct for a refresh. Problems at this point
+ * are internal errors, so elog is sufficient.
+ */
+ if (matviewRel->rd_rel->relhasrules == false ||
+ matviewRel->rd_rules->numLocks < 1)
+ elog(ERROR,
+ "materialized view \"%s\" is missing rewrite information",
+ RelationGetRelationName(matviewRel));
+
+ if (matviewRel->rd_rules->numLocks > 1)
+ elog(ERROR,
+ "materialized view \"%s\" has too many rules",
+ RelationGetRelationName(matviewRel));
+
+ rule = matviewRel->rd_rules->rules[0];
+ if (rule->event != CMD_SELECT || !(rule->isInstead))
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
+ RelationGetRelationName(matviewRel));
+
+ actions = rule->actions;
+ if (list_length(actions) != 1)
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a single action",
+ RelationGetRelationName(matviewRel));
+
+ /*
+ * Check that there is a unique index with no WHERE clause on one or more
+ * columns of the materialized view if CONCURRENTLY is specified.
+ */
+ if (stmt->concurrent)
+ {
+ List *indexoidlist = RelationGetIndexList(matviewRel);
+ ListCell *indexoidscan;
+ bool hasUniqueIndex = false;
+
+ foreach(indexoidscan, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(indexoidscan);
+ Relation indexRel;
+
+ indexRel = index_open(indexoid, AccessShareLock);
+ hasUniqueIndex = is_usable_unique_index(indexRel);
+ index_close(indexRel, AccessShareLock);
+ if (hasUniqueIndex)
+ break;
+ }
+
+ list_free(indexoidlist);
+
+ if (!hasUniqueIndex)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot refresh materialized view \"%s\" concurrently",
+ quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
+ RelationGetRelationName(matviewRel))),
+ errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
+ }
+
+ /*
+ * The stored query was rewritten at the time of the MV definition, but
+ * has not been scribbled on by the planner.
+ */
+ dataQuery = linitial_node(Query, actions);
+
+ /*
+ * Check for active uses of the relation in the current transaction, such
+ * as open scans.
+ *
+ * NB: We count on this to protect us against problems with refreshing the
+ * data using TABLE_INSERT_FROZEN.
+ */
+ CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+
+ *rel = matviewRel;
+ *objectId = matviewOid;
+
+ return dataQuery;
+}
+
+/*
+ * get_new_heap_oid
+ *
+ * Create a new heap and return its oid to which the refresh materialized view
+ * data is inserted into.
+ */
+static Oid
+get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid matviewOid,
+ char *relpersistence)
+{
+ Oid OIDNewHeap;
+ bool concurrent;
+ Oid tableSpace;
+
+ concurrent = stmt->concurrent;
+
+ /* Concurrent refresh builds new data in temp tablespace, and does diff. */
+ if (concurrent)
+ {
+ tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
+ *relpersistence = RELPERSISTENCE_TEMP;
+ }
+ else
+ {
+ tableSpace = matviewRel->rd_rel->reltablespace;
+ *relpersistence = matviewRel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Create the transient table that will receive the regenerated data. Lock
+ * it against access by any other process until commit (by which time it
+ * will be gone).
+ */
+ OIDNewHeap = make_new_heap(matviewOid, tableSpace, *relpersistence,
+ ExclusiveLock);
+ LockRelationOid(OIDNewHeap, AccessExclusiveLock);
+
+ return OIDNewHeap;
+}
+
+/*
+ * match_matview_with_new_data
+ *
+ * Arrange the materialized view newly generated data to match the existing
+ * data i.e merge in case of CONCURRENTLY otherwise perform heap swap and
+ * truncate the materialized view.
+ */
+static void
+match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, Oid OIDNewHeap, Oid relowner,
+ int save_sec_context, char relpersistence,
+ uint64 processed)
+{
+ /* Make the materialized view match the newly generated data. */
+ if (stmt->concurrent)
+ {
+ int old_depth = matview_maintenance_depth;
+
+ PG_TRY();
+ {
+ refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
+ save_sec_context);
+ }
+ PG_CATCH();
+ {
+ matview_maintenance_depth = old_depth;
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ Assert(matview_maintenance_depth == old_depth);
+ }
+ else
+ {
+ refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
+
+ /*
+ * Inform stats collector about our activity: basically, we truncated
+ * the materialized view and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the inserts
+ * and deletes it issues get counted by lower-level code.)
+ */
+ pgstat_count_truncate(matviewRel);
+ if (!stmt->skipData)
+ pgstat_count_heap_insert(matviewRel, processed);
+ }
+}
+
+/*
+ * rewrite_refresh_matview_query
+ *
+ * Rewrite the refresh materialized view data generating query.
+ *
+ * Work on the copied query to preserve the original query. Because the
+ * rewriter and planner tend to scribble on the input, we make a preliminary
+ * copy of the source querytree. This prevents problems in the case that
+ * REFRESH MATERIALIZED VIEW is in a portal or plpgsql function and is executed
+ * repeatedly. (See also the same hack in EXPLAIN and PREPARE.)
+ */
+static Query *
+rewrite_refresh_matview_query(Query *dataQuery)
+{
+ List *rewritten;
+ Query *copied_query;
+
+ /* Lock and rewrite, using a copy to preserve the original query. */
+ copied_query = copyObject(dataQuery);
+ AcquireRewriteLocks(copied_query, true, false);
+ rewritten = QueryRewrite(copied_query);
+
+ /* SELECT should never rewrite to more or less than one SELECT query */
+ if (list_length(rewritten) != 1)
+ elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
+ dataQuery = (Query *) linitial(rewritten);
+
+ return dataQuery;
+}
--
2.25.1
v1-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchapplication/x-patch; name=v1-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchDownload
From 514aa9a4b84b4e72c74fc40cd8d9fe7169d75885 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Sat, 26 Dec 2020 10:20:48 +0530
Subject: [PATCH v1] EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
Currently, explain/explain analyze refresh materialized view(RMV)
is not allowed. We do plan the materialized view query before every
refresh. I propose to show the explain/explain analyze of the
select part of the materialized view. It will be useful for the
user to know what exactly is being planned and executed as part of
RMV. Please note that we already have explain/explain analyze
CTAS/Create Mat View(CMV), where we show the explain/explain analyze
of the select part. This proposal will do the same thing.
The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.
---
src/backend/commands/explain.c | 50 +++++++++++-----
src/backend/commands/matview.c | 36 +++++++++---
src/backend/commands/prepare.c | 3 +-
src/backend/tcop/utility.c | 3 +-
src/include/commands/explain.h | 30 +++++++++-
src/include/commands/matview.h | 8 ++-
src/test/regress/expected/matview.out | 83 +++++++++++++++++++++++++++
src/test/regress/sql/matview.sql | 27 +++++++++
8 files changed, 215 insertions(+), 25 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 43f9b01e83..0189350416 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -17,6 +17,7 @@
#include "catalog/pg_type.h"
#include "commands/createas.h"
#include "commands/defrem.h"
+#include "commands/matview.h"
#include "commands/prepare.h"
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
@@ -53,10 +54,6 @@ explain_get_index_name_hook_type explain_get_index_name_hook = NULL;
#define X_CLOSE_IMMEDIATE 2
#define X_NOWHITESPACE 4
-static void ExplainOneQuery(Query *query, int cursorOptions,
- IntoClause *into, ExplainState *es,
- const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv);
static void ExplainPrintJIT(ExplainState *es, int jit_flags,
JitInstrumentation *ji);
static void report_triggers(ResultRelInfo *rInfo, bool show_relname,
@@ -274,7 +271,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
{
ExplainOneQuery(lfirst_node(Query, l),
CURSOR_OPT_PARALLEL_OK, NULL, es,
- pstate->p_sourcetext, params, pstate->p_queryEnv);
+ pstate->p_sourcetext, params, pstate->p_queryEnv,
+ NULL);
/* Separate plans with an appropriate separator */
if (lnext(rewritten, l) != NULL)
@@ -357,11 +355,11 @@ ExplainResultDesc(ExplainStmt *stmt)
*
* "into" is NULL unless we are explaining the contents of a CreateTableAsStmt.
*/
-static void
+void
ExplainOneQuery(Query *query, int cursorOptions,
IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv)
+ QueryEnvironment *queryEnv, RefreshMatViewInfo *matviewInfo)
{
/* planner will not cope with utility statements */
if (query->commandType == CMD_UTILITY)
@@ -402,7 +400,8 @@ ExplainOneQuery(Query *query, int cursorOptions,
/* run it (if needed) and produce output */
ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ matviewInfo);
}
}
@@ -439,7 +438,7 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
CURSOR_OPT_PARALLEL_OK, ctas->into, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv, NULL);
}
else if (IsA(utilityStmt, DeclareCursorStmt))
{
@@ -458,7 +457,8 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
dcs->options, NULL, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv,
+ NULL);
}
else if (IsA(utilityStmt, ExecuteStmt))
ExplainExecuteQuery((ExecuteStmt *) utilityStmt, into, es,
@@ -470,6 +470,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
else
ExplainDummyGroup("Notify", NULL, es);
}
+ else if(IsA(utilityStmt, RefreshMatViewStmt))
+ {
+ RefreshMatViewExplainInfo explainInfo;
+
+ explainInfo.es = es;
+ explainInfo.queryEnv = queryEnv;
+
+ ExecRefreshMatView((RefreshMatViewStmt *) utilityStmt,
+ queryString, params, NULL, &explainInfo);
+ }
else
{
if (es->format == EXPLAIN_FORMAT_TEXT)
@@ -496,7 +506,7 @@ void
ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
- const BufferUsage *bufusage)
+ const BufferUsage *bufusage, RefreshMatViewInfo *matviewInfo)
{
DestReceiver *dest;
QueryDesc *queryDesc;
@@ -537,6 +547,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
if (into)
dest = CreateIntoRelDestReceiver(into);
+ else if(matviewInfo)
+ dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);
else
dest = None_Receiver;
@@ -561,8 +573,12 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
{
ScanDirection dir;
- /* EXPLAIN ANALYZE CREATE TABLE AS WITH NO DATA is weird */
- if (into && into->skipData)
+ /*
+ * EXPLAIN ANALYZE CREATE TABLE AS or REFRESH MATERIALIZED VIEW
+ * WITH NO DATA is weird.
+ */
+ if ((into && into->skipData) ||
+ (matviewInfo && matviewInfo->skipData))
dir = NoMovementScanDirection;
else
dir = ForwardScanDirection;
@@ -570,6 +586,14 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
/* run the plan */
ExecutorRun(queryDesc, dir, 0L, true);
+ /*
+ * Collect the number of rows inserted in case of REFRESH MATERIALIZED
+ * VIEW which will be used while merging the newly generated data with
+ * the existing materialized view data in ExecRefreshMatView.
+ */
+ if (matviewInfo)
+ matviewInfo->processed = queryDesc->estate->es_processed;
+
/* run cleanup too */
ExecutorFinish(queryDesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 40cb436d16..53af40d4fa 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -146,7 +146,8 @@ SetMatViewPopulatedState(Relation relation, bool newstate)
*/
ObjectAddress
ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc)
+ ParamListInfo params, QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo)
{
Oid matviewOid;
Relation matviewRel;
@@ -184,8 +185,11 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
- &relpersistence);
+ if (explainInfo && !explainInfo->es->analyze)
+ OIDNewHeap = InvalidOid;
+ else
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -194,17 +198,35 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_RESTRICTED_OPERATION);
/* Generate the data, if wanted. */
- if (!stmt->skipData)
+ if (!stmt->skipData && !explainInfo)
{
dataQuery = rewrite_refresh_matview_query(dataQuery);
processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
queryString);
}
+ else if (explainInfo)
+ {
+ RefreshMatViewInfo matViewInfo;
+
+ matViewInfo.OIDNewHeap = OIDNewHeap;
+ matViewInfo.skipData = stmt->skipData;
+ matViewInfo.processed = 0;
+
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
+
+ ExplainOneQuery(dataQuery,
+ CURSOR_OPT_PARALLEL_OK, NULL, explainInfo->es,
+ queryString, params, explainInfo->queryEnv,
+ &matViewInfo);
+
+ processed = matViewInfo.processed;
+ }
- match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
- relowner, save_sec_context, relpersistence,
- processed);
+ if (OidIsValid(OIDNewHeap))
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 89087a7be3..07166479e7 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -672,7 +672,8 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
if (pstmt->commandType != CMD_UTILITY)
ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ NULL);
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index a42ead7d69..3ffc6d180f 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1659,7 +1659,8 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
address = ExecRefreshMatView((RefreshMatViewStmt *) parsetree,
- queryString, params, qc);
+ queryString, params, qc,
+ NULL);
}
PG_FINALLY();
{
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index ba661d32a6..eccc3378b7 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -61,6 +61,27 @@ typedef struct ExplainState
ExplainWorkersState *workers_state; /* needed if parallel plan */
} ExplainState;
+/*
+ * Refresh Materialized View information passed across functions for EXPLAIN
+ * execution.
+ */
+typedef struct RefreshMatViewInfo
+{
+ /* Oid of the new heap created. */
+ Oid OIDNewHeap;
+ /* Is WITH NO DATA clause specified? */
+ bool skipData;
+ /* Number of rows inserted. */
+ uint64 processed;
+} RefreshMatViewInfo;
+
+/* EXPLAIN information shared to ExecRefreshMatView(). */
+typedef struct RefreshMatViewExplainInfo
+{
+ ExplainState *es;
+ QueryEnvironment *queryEnv;
+} RefreshMatViewExplainInfo;
+
/* Hook for plugins to get control in ExplainOneQuery() */
typedef void (*ExplainOneQuery_hook_type) (Query *query,
int cursorOptions,
@@ -91,7 +112,14 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
- const BufferUsage *bufusage);
+ const BufferUsage *bufusage,
+ RefreshMatViewInfo *matviewInfo);
+
+extern void ExplainOneQuery(Query *query, int cursorOptions,
+ IntoClause *into, ExplainState *es,
+ const char *queryString, ParamListInfo params,
+ QueryEnvironment *queryEnv,
+ RefreshMatViewInfo *matviewInfo);
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
diff --git a/src/include/commands/matview.h b/src/include/commands/matview.h
index 3ea4f5c80b..ad3554cac8 100644
--- a/src/include/commands/matview.h
+++ b/src/include/commands/matview.h
@@ -15,6 +15,7 @@
#define MATVIEW_H
#include "catalog/objectaddress.h"
+#include "commands/explain.h"
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "tcop/dest.h"
@@ -23,8 +24,11 @@
extern void SetMatViewPopulatedState(Relation relation, bool newstate);
-extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc);
+extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt,
+ const char *queryString,
+ ParamListInfo params,
+ QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo);
extern DestReceiver *CreateTransientRelDestReceiver(Oid oid);
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 2c0760404d..0287917be8 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -630,3 +630,86 @@ drop cascades to materialized view matview_schema.mv_withdata2
drop cascades to materialized view matview_schema.mv_nodata1
drop cascades to materialized view matview_schema.mv_nodata2
DROP USER regress_matview_user;
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+-----------------------------------------
+ Seq Scan on mv_exp_tbl (never executed)
+ Filter: (a > 5)
+(2 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+ERROR: materialized view "mv_exp" has not been populated
+HINT: Use the REFRESH MATERIALIZED VIEW command.
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
diff --git a/src/test/regress/sql/matview.sql b/src/test/regress/sql/matview.sql
index 70c4954d89..c13a7773aa 100644
--- a/src/test/regress/sql/matview.sql
+++ b/src/test/regress/sql/matview.sql
@@ -264,3 +264,30 @@ ALTER DEFAULT PRIVILEGES FOR ROLE regress_matview_user
DROP SCHEMA matview_schema CASCADE;
DROP USER regress_matview_user;
+
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
--
2.25.1
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Mon, Dec 28, 2020 at 5:56 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
On Tue, Dec 22, 2020 at 7:01 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:Currently, $subject is not allowed. We do plan the mat view query
before every refresh. I propose to show the explain/explain analyze of
the select part of the mat view in case of Refresh Mat View(RMV). It
will be useful for the user to know what exactly is being planned and
executed as part of RMV. Please note that we already have
explain/explain analyze CTAS/Create Mat View(CMV), where we show the
explain/explain analyze of the select part. This proposal will do the
same thing.The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.Thoughts? If okay, I will post a patch later.
Attaching below patches:
0001 - Rearrange Refresh Mat View Code - Currently, the function
ExecRefreshMatView in matview.c is having many lines of code which is
not at all good from readability and maintainability perspectives.
This patch adds a few functions and moves the code from
ExecRefreshMatView to them making the code look better.0002 - EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW support and tests.
If this proposal is useful, I have few open points - 1) In the patch I
have added a new mat view info parameter to ExplainOneQuery(), do we
also need to add it to ExplainOneQuery_hook_type? 2) Do we document
(under respective command pages or somewhere else) that we allow
explain/explain analyze for a command?Thoughts?
Attaching v2 patch set reabsed on the latest master f7a1a805cb. And
also added an entry for upcoming commitfest -
https://commitfest.postgresql.org/32/2928/
Please consider the v2 patches for further review.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v2-0001-Rearrange-Refresh-Mat-View-Code.patchapplication/octet-stream; name=v2-0001-Rearrange-Refresh-Mat-View-Code.patchDownload
From d0e050867e256dc6ff64aae01a4f4a50e0872c38 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Thu, 7 Jan 2021 14:42:33 +0530
Subject: [PATCH v2 1/1] Rearrange Refresh Mat View Code
Currently, the function ExecRefreshMatView in matview.c is having
many lines of code which is not at all good from readability and
maintainability perspectives. This patch adds few functions and
moves the code from ExecRefreshMatView to them making the code
look better.
---
src/backend/commands/matview.c | 452 ++++++++++++++++++++-------------
1 file changed, 273 insertions(+), 179 deletions(-)
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..034536fd0c 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -64,7 +64,7 @@ static void transientrel_startup(DestReceiver *self, int operation, TupleDesc ty
static bool transientrel_receive(TupleTableSlot *slot, DestReceiver *self);
static void transientrel_shutdown(DestReceiver *self);
static void transientrel_destroy(DestReceiver *self);
-static uint64 refresh_matview_datafill(DestReceiver *dest, Query *query,
+static uint64 refresh_matview_datafill(Oid OIDNewHeap, Query *query,
const char *queryString);
static char *make_temptable_name_n(char *tempname, int n);
static void refresh_by_match_merge(Oid matviewOid, Oid tempOid, Oid relowner,
@@ -73,6 +73,16 @@ static void refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersist
static bool is_usable_unique_index(Relation indexRel);
static void OpenMatViewIncrementalMaintenance(void);
static void CloseMatViewIncrementalMaintenance(void);
+static Query *get_matview_query(RefreshMatViewStmt *stmt, Relation *rel,
+ Oid *objectId);
+static Query *rewrite_refresh_matview_query(Query *dataQuery);
+static Oid get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, char *relpersistence);
+static void match_matview_with_new_data(RefreshMatViewStmt *stmt,
+ Relation matviewRel, Oid matviewOid,
+ Oid OIDNewHeap, Oid relowner,
+ int save_sec_context,
+ char relpersistence, uint64 processed);
/*
* SetMatViewPopulatedState
@@ -140,127 +150,20 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
{
Oid matviewOid;
Relation matviewRel;
- RewriteRule *rule;
- List *actions;
Query *dataQuery;
- Oid tableSpace;
- Oid relowner;
Oid OIDNewHeap;
- DestReceiver *dest;
uint64 processed = 0;
- bool concurrent;
- LOCKMODE lockmode;
+ Oid relowner;
char relpersistence;
Oid save_userid;
int save_sec_context;
int save_nestlevel;
ObjectAddress address;
- /* Determine strength of lock needed. */
- concurrent = stmt->concurrent;
- lockmode = concurrent ? ExclusiveLock : AccessExclusiveLock;
-
- /*
- * Get a lock until end of transaction.
- */
- matviewOid = RangeVarGetRelidExtended(stmt->relation,
- lockmode, 0,
- RangeVarCallbackOwnsTable, NULL);
- matviewRel = table_open(matviewOid, NoLock);
-
- /* Make sure it is a materialized view. */
- if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("\"%s\" is not a materialized view",
- RelationGetRelationName(matviewRel))));
-
- /* Check that CONCURRENTLY is not specified if not populated. */
- if (concurrent && !RelationIsPopulated(matviewRel))
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
-
- /* Check that conflicting options have not been specified. */
- if (concurrent && stmt->skipData)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
-
- /*
- * Check that everything is correct for a refresh. Problems at this point
- * are internal errors, so elog is sufficient.
- */
- if (matviewRel->rd_rel->relhasrules == false ||
- matviewRel->rd_rules->numLocks < 1)
- elog(ERROR,
- "materialized view \"%s\" is missing rewrite information",
- RelationGetRelationName(matviewRel));
-
- if (matviewRel->rd_rules->numLocks > 1)
- elog(ERROR,
- "materialized view \"%s\" has too many rules",
- RelationGetRelationName(matviewRel));
-
- rule = matviewRel->rd_rules->rules[0];
- if (rule->event != CMD_SELECT || !(rule->isInstead))
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
- RelationGetRelationName(matviewRel));
-
- actions = rule->actions;
- if (list_length(actions) != 1)
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a single action",
- RelationGetRelationName(matviewRel));
-
- /*
- * Check that there is a unique index with no WHERE clause on one or more
- * columns of the materialized view if CONCURRENTLY is specified.
- */
- if (concurrent)
- {
- List *indexoidlist = RelationGetIndexList(matviewRel);
- ListCell *indexoidscan;
- bool hasUniqueIndex = false;
-
- foreach(indexoidscan, indexoidlist)
- {
- Oid indexoid = lfirst_oid(indexoidscan);
- Relation indexRel;
-
- indexRel = index_open(indexoid, AccessShareLock);
- hasUniqueIndex = is_usable_unique_index(indexRel);
- index_close(indexRel, AccessShareLock);
- if (hasUniqueIndex)
- break;
- }
-
- list_free(indexoidlist);
-
- if (!hasUniqueIndex)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot refresh materialized view \"%s\" concurrently",
- quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
- RelationGetRelationName(matviewRel))),
- errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
- }
-
- /*
- * The stored query was rewritten at the time of the MV definition, but
- * has not been scribbled on by the planner.
- */
- dataQuery = linitial_node(Query, actions);
+ /* Get the data generating query. */
+ dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);
- /*
- * Check for active uses of the relation in the current transaction, such
- * as open scans.
- *
- * NB: We count on this to protect us against problems with refreshing the
- * data using TABLE_INSERT_FROZEN.
- */
- CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+ relowner = matviewRel->rd_rel->relowner;
/*
* Tentatively mark the matview as populated or not (this will roll back
@@ -281,27 +184,8 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- /* Concurrent refresh builds new data in temp tablespace, and does diff. */
- if (concurrent)
- {
- tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
- relpersistence = RELPERSISTENCE_TEMP;
- }
- else
- {
- tableSpace = matviewRel->rd_rel->reltablespace;
- relpersistence = matviewRel->rd_rel->relpersistence;
- }
-
- /*
- * Create the transient table that will receive the regenerated data. Lock
- * it against access by any other process until commit (by which time it
- * will be gone).
- */
- OIDNewHeap = make_new_heap(matviewOid, tableSpace, relpersistence,
- ExclusiveLock);
- LockRelationOid(OIDNewHeap, AccessExclusiveLock);
- dest = CreateTransientRelDestReceiver(OIDNewHeap);
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -311,40 +195,16 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/* Generate the data, if wanted. */
if (!stmt->skipData)
- processed = refresh_matview_datafill(dest, dataQuery, queryString);
-
- /* Make the matview match the newly generated data. */
- if (concurrent)
{
- int old_depth = matview_maintenance_depth;
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
- PG_TRY();
- {
- refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
- save_sec_context);
- }
- PG_CATCH();
- {
- matview_maintenance_depth = old_depth;
- PG_RE_THROW();
- }
- PG_END_TRY();
- Assert(matview_maintenance_depth == old_depth);
+ processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
+ queryString);
}
- else
- {
- refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
- /*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
- */
- pgstat_count_truncate(matviewRel);
- if (!stmt->skipData)
- pgstat_count_heap_insert(matviewRel, processed);
- }
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
@@ -373,30 +233,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/*
* refresh_matview_datafill
*
- * Execute the given query, sending result rows to "dest" (which will
- * insert them into the target matview).
+ * Create dest receiver and execute the given query, sending result rows to the
+ * dest receiver which will insert them into the target materialized view.
*
* Returns number of rows inserted.
*/
static uint64
-refresh_matview_datafill(DestReceiver *dest, Query *query,
- const char *queryString)
+refresh_matview_datafill(Oid OIDNewHeap, Query *query, const char *queryString)
{
- List *rewritten;
PlannedStmt *plan;
QueryDesc *queryDesc;
- Query *copied_query;
uint64 processed;
-
- /* Lock and rewrite, using a copy to preserve the original query. */
- copied_query = copyObject(query);
- AcquireRewriteLocks(copied_query, true, false);
- rewritten = QueryRewrite(copied_query);
-
- /* SELECT should never rewrite to more or less than one SELECT query */
- if (list_length(rewritten) != 1)
- elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
- query = (Query *) linitial(rewritten);
+ DestReceiver *dest;
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
@@ -413,6 +261,8 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
PushCopiedSnapshot(GetActiveSnapshot());
UpdateActiveSnapshotCommandId();
+ dest = CreateTransientRelDestReceiver(OIDNewHeap);
+
/* Create a QueryDesc, redirecting output to our tuple receiver */
queryDesc = CreateQueryDesc(plan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
@@ -933,3 +783,247 @@ CloseMatViewIncrementalMaintenance(void)
matview_maintenance_depth--;
Assert(matview_maintenance_depth >= 0);
}
+
+/*
+ * get_matview_query
+ *
+ * Open the refresh materialized view relation, perform sanity checks and also
+ * get the associated data generating query from it.
+ *
+ * Note that the refresh materialized view relation is opened here, it has to
+ * be closed in the caller.
+ */
+static Query *
+get_matview_query(RefreshMatViewStmt *stmt, Relation *rel, Oid *objectId)
+{
+ Oid matviewOid;
+ Relation matviewRel;
+ RewriteRule *rule;
+ List *actions;
+ Query *dataQuery;
+ LOCKMODE lockmode;
+
+ /* Determine strength of lock needed. */
+ lockmode = stmt->concurrent ? ExclusiveLock : AccessExclusiveLock;
+
+ /* Get a lock until end of transaction. */
+ matviewOid = RangeVarGetRelidExtended(stmt->relation, lockmode, 0,
+ RangeVarCallbackOwnsTable, NULL);
+ matviewRel = table_open(matviewOid, NoLock);
+
+ /* Make sure it is a materialized view. */
+ if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("\"%s\" is not a materialized view",
+ RelationGetRelationName(matviewRel))));
+
+ /* Check that CONCURRENTLY is not specified if not populated. */
+ if (stmt->concurrent && !RelationIsPopulated(matviewRel))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
+
+ /* Check that conflicting options have not been specified. */
+ if (stmt->concurrent && stmt->skipData)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
+
+ /*
+ * Check that everything is correct for a refresh. Problems at this point
+ * are internal errors, so elog is sufficient.
+ */
+ if (matviewRel->rd_rel->relhasrules == false ||
+ matviewRel->rd_rules->numLocks < 1)
+ elog(ERROR,
+ "materialized view \"%s\" is missing rewrite information",
+ RelationGetRelationName(matviewRel));
+
+ if (matviewRel->rd_rules->numLocks > 1)
+ elog(ERROR,
+ "materialized view \"%s\" has too many rules",
+ RelationGetRelationName(matviewRel));
+
+ rule = matviewRel->rd_rules->rules[0];
+ if (rule->event != CMD_SELECT || !(rule->isInstead))
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
+ RelationGetRelationName(matviewRel));
+
+ actions = rule->actions;
+ if (list_length(actions) != 1)
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a single action",
+ RelationGetRelationName(matviewRel));
+
+ /*
+ * Check that there is a unique index with no WHERE clause on one or more
+ * columns of the materialized view if CONCURRENTLY is specified.
+ */
+ if (stmt->concurrent)
+ {
+ List *indexoidlist = RelationGetIndexList(matviewRel);
+ ListCell *indexoidscan;
+ bool hasUniqueIndex = false;
+
+ foreach(indexoidscan, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(indexoidscan);
+ Relation indexRel;
+
+ indexRel = index_open(indexoid, AccessShareLock);
+ hasUniqueIndex = is_usable_unique_index(indexRel);
+ index_close(indexRel, AccessShareLock);
+ if (hasUniqueIndex)
+ break;
+ }
+
+ list_free(indexoidlist);
+
+ if (!hasUniqueIndex)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot refresh materialized view \"%s\" concurrently",
+ quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
+ RelationGetRelationName(matviewRel))),
+ errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
+ }
+
+ /*
+ * The stored query was rewritten at the time of the MV definition, but
+ * has not been scribbled on by the planner.
+ */
+ dataQuery = linitial_node(Query, actions);
+
+ /*
+ * Check for active uses of the relation in the current transaction, such
+ * as open scans.
+ *
+ * NB: We count on this to protect us against problems with refreshing the
+ * data using TABLE_INSERT_FROZEN.
+ */
+ CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+
+ *rel = matviewRel;
+ *objectId = matviewOid;
+
+ return dataQuery;
+}
+
+/*
+ * get_new_heap_oid
+ *
+ * Create a new heap and return its oid to which the refresh materialized view
+ * data is inserted into.
+ */
+static Oid
+get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid matviewOid,
+ char *relpersistence)
+{
+ Oid OIDNewHeap;
+ bool concurrent;
+ Oid tableSpace;
+
+ concurrent = stmt->concurrent;
+
+ /* Concurrent refresh builds new data in temp tablespace, and does diff. */
+ if (concurrent)
+ {
+ tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
+ *relpersistence = RELPERSISTENCE_TEMP;
+ }
+ else
+ {
+ tableSpace = matviewRel->rd_rel->reltablespace;
+ *relpersistence = matviewRel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Create the transient table that will receive the regenerated data. Lock
+ * it against access by any other process until commit (by which time it
+ * will be gone).
+ */
+ OIDNewHeap = make_new_heap(matviewOid, tableSpace, *relpersistence,
+ ExclusiveLock);
+ LockRelationOid(OIDNewHeap, AccessExclusiveLock);
+
+ return OIDNewHeap;
+}
+
+/*
+ * match_matview_with_new_data
+ *
+ * Arrange the materialized view newly generated data to match the existing
+ * data i.e merge in case of CONCURRENTLY otherwise perform heap swap and
+ * truncate the materialized view.
+ */
+static void
+match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, Oid OIDNewHeap, Oid relowner,
+ int save_sec_context, char relpersistence,
+ uint64 processed)
+{
+ /* Make the materialized view match the newly generated data. */
+ if (stmt->concurrent)
+ {
+ int old_depth = matview_maintenance_depth;
+
+ PG_TRY();
+ {
+ refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
+ save_sec_context);
+ }
+ PG_CATCH();
+ {
+ matview_maintenance_depth = old_depth;
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ Assert(matview_maintenance_depth == old_depth);
+ }
+ else
+ {
+ refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
+
+ /*
+ * Inform stats collector about our activity: basically, we truncated
+ * the materialized view and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the inserts
+ * and deletes it issues get counted by lower-level code.)
+ */
+ pgstat_count_truncate(matviewRel);
+ if (!stmt->skipData)
+ pgstat_count_heap_insert(matviewRel, processed);
+ }
+}
+
+/*
+ * rewrite_refresh_matview_query
+ *
+ * Rewrite the refresh materialized view data generating query.
+ *
+ * Work on the copied query to preserve the original query. Because the
+ * rewriter and planner tend to scribble on the input, we make a preliminary
+ * copy of the source querytree. This prevents problems in the case that
+ * REFRESH MATERIALIZED VIEW is in a portal or plpgsql function and is executed
+ * repeatedly. (See also the same hack in EXPLAIN and PREPARE.)
+ */
+static Query *
+rewrite_refresh_matview_query(Query *dataQuery)
+{
+ List *rewritten;
+ Query *copied_query;
+
+ /* Lock and rewrite, using a copy to preserve the original query. */
+ copied_query = copyObject(dataQuery);
+ AcquireRewriteLocks(copied_query, true, false);
+ rewritten = QueryRewrite(copied_query);
+
+ /* SELECT should never rewrite to more or less than one SELECT query */
+ if (list_length(rewritten) != 1)
+ elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
+ dataQuery = (Query *) linitial(rewritten);
+
+ return dataQuery;
+}
--
2.25.1
v2-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchapplication/octet-stream; name=v2-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchDownload
From 4895dce8bb6af574ba16b1cebc502a6b35e76c41 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Thu, 7 Jan 2021 15:16:41 +0530
Subject: [PATCH v2 2/2] EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
Currently, explain/explain analyze refresh materialized view(RMV)
is not allowed. We do plan the materialized view query before every
refresh. I propose to show the explain/explain analyze of the
select part of the materialized view. It will be useful for the
user to know what exactly is being planned and executed as part of
RMV. Please note that we already have explain/explain analyze
CTAS/Create Mat View(CMV), where we show the explain/explain analyze
of the select part. This proposal will do the same thing.
The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.
---
src/backend/commands/explain.c | 50 +++++++++++-----
src/backend/commands/matview.c | 36 +++++++++---
src/backend/commands/prepare.c | 3 +-
src/backend/tcop/utility.c | 3 +-
src/include/commands/explain.h | 30 +++++++++-
src/include/commands/matview.h | 8 ++-
src/test/regress/expected/matview.out | 83 +++++++++++++++++++++++++++
src/test/regress/sql/matview.sql | 27 +++++++++
8 files changed, 215 insertions(+), 25 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5d7eb3574c..8373806920 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -17,6 +17,7 @@
#include "catalog/pg_type.h"
#include "commands/createas.h"
#include "commands/defrem.h"
+#include "commands/matview.h"
#include "commands/prepare.h"
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
@@ -53,10 +54,6 @@ explain_get_index_name_hook_type explain_get_index_name_hook = NULL;
#define X_CLOSE_IMMEDIATE 2
#define X_NOWHITESPACE 4
-static void ExplainOneQuery(Query *query, int cursorOptions,
- IntoClause *into, ExplainState *es,
- const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv);
static void ExplainPrintJIT(ExplainState *es, int jit_flags,
JitInstrumentation *ji);
static void report_triggers(ResultRelInfo *rInfo, bool show_relname,
@@ -274,7 +271,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
{
ExplainOneQuery(lfirst_node(Query, l),
CURSOR_OPT_PARALLEL_OK, NULL, es,
- pstate->p_sourcetext, params, pstate->p_queryEnv);
+ pstate->p_sourcetext, params, pstate->p_queryEnv,
+ NULL);
/* Separate plans with an appropriate separator */
if (lnext(rewritten, l) != NULL)
@@ -357,11 +355,11 @@ ExplainResultDesc(ExplainStmt *stmt)
*
* "into" is NULL unless we are explaining the contents of a CreateTableAsStmt.
*/
-static void
+void
ExplainOneQuery(Query *query, int cursorOptions,
IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv)
+ QueryEnvironment *queryEnv, RefreshMatViewInfo *matviewInfo)
{
/* planner will not cope with utility statements */
if (query->commandType == CMD_UTILITY)
@@ -402,7 +400,8 @@ ExplainOneQuery(Query *query, int cursorOptions,
/* run it (if needed) and produce output */
ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ matviewInfo);
}
}
@@ -455,7 +454,7 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
CURSOR_OPT_PARALLEL_OK, ctas->into, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv, NULL);
}
else if (IsA(utilityStmt, DeclareCursorStmt))
{
@@ -474,7 +473,8 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
dcs->options, NULL, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv,
+ NULL);
}
else if (IsA(utilityStmt, ExecuteStmt))
ExplainExecuteQuery((ExecuteStmt *) utilityStmt, into, es,
@@ -486,6 +486,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
else
ExplainDummyGroup("Notify", NULL, es);
}
+ else if(IsA(utilityStmt, RefreshMatViewStmt))
+ {
+ RefreshMatViewExplainInfo explainInfo;
+
+ explainInfo.es = es;
+ explainInfo.queryEnv = queryEnv;
+
+ ExecRefreshMatView((RefreshMatViewStmt *) utilityStmt,
+ queryString, params, NULL, &explainInfo);
+ }
else
{
if (es->format == EXPLAIN_FORMAT_TEXT)
@@ -512,7 +522,7 @@ void
ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
- const BufferUsage *bufusage)
+ const BufferUsage *bufusage, RefreshMatViewInfo *matviewInfo)
{
DestReceiver *dest;
QueryDesc *queryDesc;
@@ -553,6 +563,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
if (into)
dest = CreateIntoRelDestReceiver(into);
+ else if(matviewInfo)
+ dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);
else
dest = None_Receiver;
@@ -577,8 +589,12 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
{
ScanDirection dir;
- /* EXPLAIN ANALYZE CREATE TABLE AS WITH NO DATA is weird */
- if (into && into->skipData)
+ /*
+ * EXPLAIN ANALYZE CREATE TABLE AS or REFRESH MATERIALIZED VIEW
+ * WITH NO DATA is weird.
+ */
+ if ((into && into->skipData) ||
+ (matviewInfo && matviewInfo->skipData))
dir = NoMovementScanDirection;
else
dir = ForwardScanDirection;
@@ -586,6 +602,14 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
/* run the plan */
ExecutorRun(queryDesc, dir, 0L, true);
+ /*
+ * Collect the number of rows inserted in case of REFRESH MATERIALIZED
+ * VIEW which will be used while merging the newly generated data with
+ * the existing materialized view data in ExecRefreshMatView.
+ */
+ if (matviewInfo)
+ matviewInfo->processed = queryDesc->estate->es_processed;
+
/* run cleanup too */
ExecutorFinish(queryDesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 034536fd0c..ac4a668ea3 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -146,7 +146,8 @@ SetMatViewPopulatedState(Relation relation, bool newstate)
*/
ObjectAddress
ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc)
+ ParamListInfo params, QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo)
{
Oid matviewOid;
Relation matviewRel;
@@ -184,8 +185,11 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
- &relpersistence);
+ if (explainInfo && !explainInfo->es->analyze)
+ OIDNewHeap = InvalidOid;
+ else
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -194,17 +198,35 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_RESTRICTED_OPERATION);
/* Generate the data, if wanted. */
- if (!stmt->skipData)
+ if (!stmt->skipData && !explainInfo)
{
dataQuery = rewrite_refresh_matview_query(dataQuery);
processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
queryString);
}
+ else if (explainInfo)
+ {
+ RefreshMatViewInfo matViewInfo;
+
+ matViewInfo.OIDNewHeap = OIDNewHeap;
+ matViewInfo.skipData = stmt->skipData;
+ matViewInfo.processed = 0;
+
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
+
+ ExplainOneQuery(dataQuery,
+ CURSOR_OPT_PARALLEL_OK, NULL, explainInfo->es,
+ queryString, params, explainInfo->queryEnv,
+ &matViewInfo);
+
+ processed = matViewInfo.processed;
+ }
- match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
- relowner, save_sec_context, relpersistence,
- processed);
+ if (OidIsValid(OIDNewHeap))
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 653ef8e41a..696d3343d4 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -672,7 +672,8 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
if (pstmt->commandType != CMD_UTILITY)
ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ NULL);
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 53a511f1da..ae7cef9c24 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1659,7 +1659,8 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
address = ExecRefreshMatView((RefreshMatViewStmt *) parsetree,
- queryString, params, qc);
+ queryString, params, qc,
+ NULL);
}
PG_FINALLY();
{
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7f27d92df1 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -61,6 +61,27 @@ typedef struct ExplainState
ExplainWorkersState *workers_state; /* needed if parallel plan */
} ExplainState;
+/*
+ * Refresh Materialized View information passed across functions for EXPLAIN
+ * execution.
+ */
+typedef struct RefreshMatViewInfo
+{
+ /* Oid of the new heap created. */
+ Oid OIDNewHeap;
+ /* Is WITH NO DATA clause specified? */
+ bool skipData;
+ /* Number of rows inserted. */
+ uint64 processed;
+} RefreshMatViewInfo;
+
+/* EXPLAIN information shared to ExecRefreshMatView(). */
+typedef struct RefreshMatViewExplainInfo
+{
+ ExplainState *es;
+ QueryEnvironment *queryEnv;
+} RefreshMatViewExplainInfo;
+
/* Hook for plugins to get control in ExplainOneQuery() */
typedef void (*ExplainOneQuery_hook_type) (Query *query,
int cursorOptions,
@@ -91,7 +112,14 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
- const BufferUsage *bufusage);
+ const BufferUsage *bufusage,
+ RefreshMatViewInfo *matviewInfo);
+
+extern void ExplainOneQuery(Query *query, int cursorOptions,
+ IntoClause *into, ExplainState *es,
+ const char *queryString, ParamListInfo params,
+ QueryEnvironment *queryEnv,
+ RefreshMatViewInfo *matviewInfo);
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
diff --git a/src/include/commands/matview.h b/src/include/commands/matview.h
index 214b1c1df6..1d60180ebc 100644
--- a/src/include/commands/matview.h
+++ b/src/include/commands/matview.h
@@ -15,6 +15,7 @@
#define MATVIEW_H
#include "catalog/objectaddress.h"
+#include "commands/explain.h"
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "tcop/dest.h"
@@ -23,8 +24,11 @@
extern void SetMatViewPopulatedState(Relation relation, bool newstate);
-extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc);
+extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt,
+ const char *queryString,
+ ParamListInfo params,
+ QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo);
extern DestReceiver *CreateTransientRelDestReceiver(Oid oid);
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..305e511fea 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -668,3 +668,86 @@ NOTICE: relation "matview_ine_tab" already exists, skipping
(0 rows)
DROP MATERIALIZED VIEW matview_ine_tab;
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+-----------------------------------------
+ Seq Scan on mv_exp_tbl (never executed)
+ Filter: (a > 5)
+(2 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+ERROR: materialized view "mv_exp" has not been populated
+HINT: Use the REFRESH MATERIALIZED VIEW command.
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
diff --git a/src/test/regress/sql/matview.sql b/src/test/regress/sql/matview.sql
index 4a4bd0d6b6..67a45bbde7 100644
--- a/src/test/regress/sql/matview.sql
+++ b/src/test/regress/sql/matview.sql
@@ -287,3 +287,30 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
CREATE MATERIALIZED VIEW IF NOT EXISTS matview_ine_tab AS
SELECT 1 / 0 WITH NO DATA; -- ok
DROP MATERIALIZED VIEW matview_ine_tab;
+
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
--
2.25.1
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Thu, 07 Jan 2021 at 17:53, Bharath Rupireddy wrote:
On Mon, Dec 28, 2020 at 5:56 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:On Tue, Dec 22, 2020 at 7:01 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:Currently, $subject is not allowed. We do plan the mat view query
before every refresh. I propose to show the explain/explain analyze of
the select part of the mat view in case of Refresh Mat View(RMV). It
will be useful for the user to know what exactly is being planned and
executed as part of RMV. Please note that we already have
explain/explain analyze CTAS/Create Mat View(CMV), where we show the
explain/explain analyze of the select part. This proposal will do the
same thing.The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.Thoughts? If okay, I will post a patch later.
Attaching below patches:
0001 - Rearrange Refresh Mat View Code - Currently, the function
ExecRefreshMatView in matview.c is having many lines of code which is
not at all good from readability and maintainability perspectives.
This patch adds a few functions and moves the code from
ExecRefreshMatView to them making the code look better.0002 - EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW support and tests.
If this proposal is useful, I have few open points - 1) In the patch I
have added a new mat view info parameter to ExplainOneQuery(), do we
also need to add it to ExplainOneQuery_hook_type? 2) Do we document
(under respective command pages or somewhere else) that we allow
explain/explain analyze for a command?Thoughts?
Attaching v2 patch set reabsed on the latest master f7a1a805cb. And
also added an entry for upcoming commitfest -
https://commitfest.postgresql.org/32/2928/Please consider the v2 patches for further review.
Thanks for updating the patch!
+ /* Get the data generating query. */
+ dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);- /*
- * Check for active uses of the relation in the current transaction, such
- * as open scans.
- *
- * NB: We count on this to protect us against problems with refreshing the
- * data using TABLE_INSERT_FROZEN.
- */
- CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+ relowner = matviewRel->rd_rel->relowner;After apply the patch, there is a duplicate
relowner = matviewRel->rd_rel->relowner;
+ else if(matviewInfo)
+ dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);If the `matviewInfo->OIDNewHeap` is invalid, IMO we don't need create
DestReceiver, isn't it? And we should add a space after `if`.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.Ltd.
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Fri, Jan 8, 2021 at 1:50 PM japin <japinli@hotmail.com> wrote:
Thanks for updating the patch!
+ /* Get the data generating query. */ + dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);- /* - * Check for active uses of the relation in the current transaction, such - * as open scans. - * - * NB: We count on this to protect us against problems with refreshing the - * data using TABLE_INSERT_FROZEN. - */ - CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW"); + relowner = matviewRel->rd_rel->relowner;After apply the patch, there is a duplicate
relowner = matviewRel->rd_rel->relowner;
Corrected that.
+ else if(matviewInfo) + dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);If the `matviewInfo->OIDNewHeap` is invalid, IMO we don't need create
DestReceiver, isn't it? And we should add a space after `if`.
Yes, we can skip creating the dest receiver when OIDNewHeap is
invalid, this can happen for plain explain refresh mat view case.
if (explainInfo && !explainInfo->es->analyze)
OIDNewHeap = InvalidOid;
else
OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
&relpersistence);
Since we don't call ExecutorRun for plain explain, we can skip the
dest receiver creation. I modified the code as below in explain.c.
if (into)
dest = CreateIntoRelDestReceiver(into);
else if (matviewInfo && OidIsValid(matviewInfo->OIDNewHeap))
dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);
else
dest = None_Receiver;
Thanks for taking a look at the patches.
Attaching v3 patches, please consider these for further review.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v3-0001-Rearrange-Refresh-Mat-View-Code.patchapplication/x-patch; name=v3-0001-Rearrange-Refresh-Mat-View-Code.patchDownload
From f6f1760e6b460d3265ee343d5b1d53f82f45fee6 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Fri, 8 Jan 2021 14:04:26 +0530
Subject: [PATCH v3 1/2] Rearrange Refresh Mat View Code
Currently, the function ExecRefreshMatView in matview.c is having
many lines of code which is not at all good from readability and
maintainability perspectives. This patch adds few functions and
moves the code from ExecRefreshMatView to them making the code
look better.
---
src/backend/commands/matview.c | 452 ++++++++++++++++++++-------------
1 file changed, 272 insertions(+), 180 deletions(-)
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..06bd5629a8 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -64,7 +64,7 @@ static void transientrel_startup(DestReceiver *self, int operation, TupleDesc ty
static bool transientrel_receive(TupleTableSlot *slot, DestReceiver *self);
static void transientrel_shutdown(DestReceiver *self);
static void transientrel_destroy(DestReceiver *self);
-static uint64 refresh_matview_datafill(DestReceiver *dest, Query *query,
+static uint64 refresh_matview_datafill(Oid OIDNewHeap, Query *query,
const char *queryString);
static char *make_temptable_name_n(char *tempname, int n);
static void refresh_by_match_merge(Oid matviewOid, Oid tempOid, Oid relowner,
@@ -73,6 +73,16 @@ static void refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersist
static bool is_usable_unique_index(Relation indexRel);
static void OpenMatViewIncrementalMaintenance(void);
static void CloseMatViewIncrementalMaintenance(void);
+static Query *get_matview_query(RefreshMatViewStmt *stmt, Relation *rel,
+ Oid *objectId);
+static Query *rewrite_refresh_matview_query(Query *dataQuery);
+static Oid get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, char *relpersistence);
+static void match_matview_with_new_data(RefreshMatViewStmt *stmt,
+ Relation matviewRel, Oid matviewOid,
+ Oid OIDNewHeap, Oid relowner,
+ int save_sec_context,
+ char relpersistence, uint64 processed);
/*
* SetMatViewPopulatedState
@@ -140,127 +150,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
{
Oid matviewOid;
Relation matviewRel;
- RewriteRule *rule;
- List *actions;
Query *dataQuery;
- Oid tableSpace;
- Oid relowner;
Oid OIDNewHeap;
- DestReceiver *dest;
uint64 processed = 0;
- bool concurrent;
- LOCKMODE lockmode;
+ Oid relowner;
char relpersistence;
Oid save_userid;
int save_sec_context;
int save_nestlevel;
ObjectAddress address;
- /* Determine strength of lock needed. */
- concurrent = stmt->concurrent;
- lockmode = concurrent ? ExclusiveLock : AccessExclusiveLock;
-
- /*
- * Get a lock until end of transaction.
- */
- matviewOid = RangeVarGetRelidExtended(stmt->relation,
- lockmode, 0,
- RangeVarCallbackOwnsTable, NULL);
- matviewRel = table_open(matviewOid, NoLock);
-
- /* Make sure it is a materialized view. */
- if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("\"%s\" is not a materialized view",
- RelationGetRelationName(matviewRel))));
-
- /* Check that CONCURRENTLY is not specified if not populated. */
- if (concurrent && !RelationIsPopulated(matviewRel))
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
-
- /* Check that conflicting options have not been specified. */
- if (concurrent && stmt->skipData)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
-
- /*
- * Check that everything is correct for a refresh. Problems at this point
- * are internal errors, so elog is sufficient.
- */
- if (matviewRel->rd_rel->relhasrules == false ||
- matviewRel->rd_rules->numLocks < 1)
- elog(ERROR,
- "materialized view \"%s\" is missing rewrite information",
- RelationGetRelationName(matviewRel));
-
- if (matviewRel->rd_rules->numLocks > 1)
- elog(ERROR,
- "materialized view \"%s\" has too many rules",
- RelationGetRelationName(matviewRel));
-
- rule = matviewRel->rd_rules->rules[0];
- if (rule->event != CMD_SELECT || !(rule->isInstead))
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
- RelationGetRelationName(matviewRel));
-
- actions = rule->actions;
- if (list_length(actions) != 1)
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a single action",
- RelationGetRelationName(matviewRel));
-
- /*
- * Check that there is a unique index with no WHERE clause on one or more
- * columns of the materialized view if CONCURRENTLY is specified.
- */
- if (concurrent)
- {
- List *indexoidlist = RelationGetIndexList(matviewRel);
- ListCell *indexoidscan;
- bool hasUniqueIndex = false;
-
- foreach(indexoidscan, indexoidlist)
- {
- Oid indexoid = lfirst_oid(indexoidscan);
- Relation indexRel;
-
- indexRel = index_open(indexoid, AccessShareLock);
- hasUniqueIndex = is_usable_unique_index(indexRel);
- index_close(indexRel, AccessShareLock);
- if (hasUniqueIndex)
- break;
- }
-
- list_free(indexoidlist);
-
- if (!hasUniqueIndex)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot refresh materialized view \"%s\" concurrently",
- quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
- RelationGetRelationName(matviewRel))),
- errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
- }
-
- /*
- * The stored query was rewritten at the time of the MV definition, but
- * has not been scribbled on by the planner.
- */
- dataQuery = linitial_node(Query, actions);
-
- /*
- * Check for active uses of the relation in the current transaction, such
- * as open scans.
- *
- * NB: We count on this to protect us against problems with refreshing the
- * data using TABLE_INSERT_FROZEN.
- */
- CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+ /* Get the data generating query. */
+ dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);
/*
* Tentatively mark the matview as populated or not (this will roll back
@@ -281,27 +182,8 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- /* Concurrent refresh builds new data in temp tablespace, and does diff. */
- if (concurrent)
- {
- tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
- relpersistence = RELPERSISTENCE_TEMP;
- }
- else
- {
- tableSpace = matviewRel->rd_rel->reltablespace;
- relpersistence = matviewRel->rd_rel->relpersistence;
- }
-
- /*
- * Create the transient table that will receive the regenerated data. Lock
- * it against access by any other process until commit (by which time it
- * will be gone).
- */
- OIDNewHeap = make_new_heap(matviewOid, tableSpace, relpersistence,
- ExclusiveLock);
- LockRelationOid(OIDNewHeap, AccessExclusiveLock);
- dest = CreateTransientRelDestReceiver(OIDNewHeap);
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -311,40 +193,16 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/* Generate the data, if wanted. */
if (!stmt->skipData)
- processed = refresh_matview_datafill(dest, dataQuery, queryString);
-
- /* Make the matview match the newly generated data. */
- if (concurrent)
{
- int old_depth = matview_maintenance_depth;
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
- PG_TRY();
- {
- refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
- save_sec_context);
- }
- PG_CATCH();
- {
- matview_maintenance_depth = old_depth;
- PG_RE_THROW();
- }
- PG_END_TRY();
- Assert(matview_maintenance_depth == old_depth);
+ processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
+ queryString);
}
- else
- {
- refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
- /*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
- */
- pgstat_count_truncate(matviewRel);
- if (!stmt->skipData)
- pgstat_count_heap_insert(matviewRel, processed);
- }
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
@@ -373,30 +231,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/*
* refresh_matview_datafill
*
- * Execute the given query, sending result rows to "dest" (which will
- * insert them into the target matview).
+ * Create dest receiver and execute the given query, sending result rows to the
+ * dest receiver which will insert them into the target materialized view.
*
* Returns number of rows inserted.
*/
static uint64
-refresh_matview_datafill(DestReceiver *dest, Query *query,
- const char *queryString)
+refresh_matview_datafill(Oid OIDNewHeap, Query *query, const char *queryString)
{
- List *rewritten;
PlannedStmt *plan;
QueryDesc *queryDesc;
- Query *copied_query;
uint64 processed;
-
- /* Lock and rewrite, using a copy to preserve the original query. */
- copied_query = copyObject(query);
- AcquireRewriteLocks(copied_query, true, false);
- rewritten = QueryRewrite(copied_query);
-
- /* SELECT should never rewrite to more or less than one SELECT query */
- if (list_length(rewritten) != 1)
- elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
- query = (Query *) linitial(rewritten);
+ DestReceiver *dest;
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
@@ -413,6 +259,8 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
PushCopiedSnapshot(GetActiveSnapshot());
UpdateActiveSnapshotCommandId();
+ dest = CreateTransientRelDestReceiver(OIDNewHeap);
+
/* Create a QueryDesc, redirecting output to our tuple receiver */
queryDesc = CreateQueryDesc(plan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
@@ -933,3 +781,247 @@ CloseMatViewIncrementalMaintenance(void)
matview_maintenance_depth--;
Assert(matview_maintenance_depth >= 0);
}
+
+/*
+ * get_matview_query
+ *
+ * Open the refresh materialized view relation, perform sanity checks and also
+ * get the associated data generating query from it.
+ *
+ * Note that the refresh materialized view relation is opened here, it has to
+ * be closed in the caller.
+ */
+static Query *
+get_matview_query(RefreshMatViewStmt *stmt, Relation *rel, Oid *objectId)
+{
+ Oid matviewOid;
+ Relation matviewRel;
+ RewriteRule *rule;
+ List *actions;
+ Query *dataQuery;
+ LOCKMODE lockmode;
+
+ /* Determine strength of lock needed. */
+ lockmode = stmt->concurrent ? ExclusiveLock : AccessExclusiveLock;
+
+ /* Get a lock until end of transaction. */
+ matviewOid = RangeVarGetRelidExtended(stmt->relation, lockmode, 0,
+ RangeVarCallbackOwnsTable, NULL);
+ matviewRel = table_open(matviewOid, NoLock);
+
+ /* Make sure it is a materialized view. */
+ if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("\"%s\" is not a materialized view",
+ RelationGetRelationName(matviewRel))));
+
+ /* Check that CONCURRENTLY is not specified if not populated. */
+ if (stmt->concurrent && !RelationIsPopulated(matviewRel))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
+
+ /* Check that conflicting options have not been specified. */
+ if (stmt->concurrent && stmt->skipData)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
+
+ /*
+ * Check that everything is correct for a refresh. Problems at this point
+ * are internal errors, so elog is sufficient.
+ */
+ if (matviewRel->rd_rel->relhasrules == false ||
+ matviewRel->rd_rules->numLocks < 1)
+ elog(ERROR,
+ "materialized view \"%s\" is missing rewrite information",
+ RelationGetRelationName(matviewRel));
+
+ if (matviewRel->rd_rules->numLocks > 1)
+ elog(ERROR,
+ "materialized view \"%s\" has too many rules",
+ RelationGetRelationName(matviewRel));
+
+ rule = matviewRel->rd_rules->rules[0];
+ if (rule->event != CMD_SELECT || !(rule->isInstead))
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
+ RelationGetRelationName(matviewRel));
+
+ actions = rule->actions;
+ if (list_length(actions) != 1)
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a single action",
+ RelationGetRelationName(matviewRel));
+
+ /*
+ * Check that there is a unique index with no WHERE clause on one or more
+ * columns of the materialized view if CONCURRENTLY is specified.
+ */
+ if (stmt->concurrent)
+ {
+ List *indexoidlist = RelationGetIndexList(matviewRel);
+ ListCell *indexoidscan;
+ bool hasUniqueIndex = false;
+
+ foreach(indexoidscan, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(indexoidscan);
+ Relation indexRel;
+
+ indexRel = index_open(indexoid, AccessShareLock);
+ hasUniqueIndex = is_usable_unique_index(indexRel);
+ index_close(indexRel, AccessShareLock);
+ if (hasUniqueIndex)
+ break;
+ }
+
+ list_free(indexoidlist);
+
+ if (!hasUniqueIndex)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot refresh materialized view \"%s\" concurrently",
+ quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
+ RelationGetRelationName(matviewRel))),
+ errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
+ }
+
+ /*
+ * The stored query was rewritten at the time of the MV definition, but
+ * has not been scribbled on by the planner.
+ */
+ dataQuery = linitial_node(Query, actions);
+
+ /*
+ * Check for active uses of the relation in the current transaction, such
+ * as open scans.
+ *
+ * NB: We count on this to protect us against problems with refreshing the
+ * data using TABLE_INSERT_FROZEN.
+ */
+ CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+
+ *rel = matviewRel;
+ *objectId = matviewOid;
+
+ return dataQuery;
+}
+
+/*
+ * get_new_heap_oid
+ *
+ * Create a new heap and return its oid to which the refresh materialized view
+ * data is inserted into.
+ */
+static Oid
+get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid matviewOid,
+ char *relpersistence)
+{
+ Oid OIDNewHeap;
+ bool concurrent;
+ Oid tableSpace;
+
+ concurrent = stmt->concurrent;
+
+ /* Concurrent refresh builds new data in temp tablespace, and does diff. */
+ if (concurrent)
+ {
+ tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
+ *relpersistence = RELPERSISTENCE_TEMP;
+ }
+ else
+ {
+ tableSpace = matviewRel->rd_rel->reltablespace;
+ *relpersistence = matviewRel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Create the transient table that will receive the regenerated data. Lock
+ * it against access by any other process until commit (by which time it
+ * will be gone).
+ */
+ OIDNewHeap = make_new_heap(matviewOid, tableSpace, *relpersistence,
+ ExclusiveLock);
+ LockRelationOid(OIDNewHeap, AccessExclusiveLock);
+
+ return OIDNewHeap;
+}
+
+/*
+ * match_matview_with_new_data
+ *
+ * Arrange the materialized view newly generated data to match the existing
+ * data i.e merge in case of CONCURRENTLY otherwise perform heap swap and
+ * truncate the materialized view.
+ */
+static void
+match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, Oid OIDNewHeap, Oid relowner,
+ int save_sec_context, char relpersistence,
+ uint64 processed)
+{
+ /* Make the materialized view match the newly generated data. */
+ if (stmt->concurrent)
+ {
+ int old_depth = matview_maintenance_depth;
+
+ PG_TRY();
+ {
+ refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
+ save_sec_context);
+ }
+ PG_CATCH();
+ {
+ matview_maintenance_depth = old_depth;
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ Assert(matview_maintenance_depth == old_depth);
+ }
+ else
+ {
+ refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
+
+ /*
+ * Inform stats collector about our activity: basically, we truncated
+ * the materialized view and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the inserts
+ * and deletes it issues get counted by lower-level code.)
+ */
+ pgstat_count_truncate(matviewRel);
+ if (!stmt->skipData)
+ pgstat_count_heap_insert(matviewRel, processed);
+ }
+}
+
+/*
+ * rewrite_refresh_matview_query
+ *
+ * Rewrite the refresh materialized view data generating query.
+ *
+ * Work on the copied query to preserve the original query. Because the
+ * rewriter and planner tend to scribble on the input, we make a preliminary
+ * copy of the source querytree. This prevents problems in the case that
+ * REFRESH MATERIALIZED VIEW is in a portal or plpgsql function and is executed
+ * repeatedly. (See also the same hack in EXPLAIN and PREPARE.)
+ */
+static Query *
+rewrite_refresh_matview_query(Query *dataQuery)
+{
+ List *rewritten;
+ Query *copied_query;
+
+ /* Lock and rewrite, using a copy to preserve the original query. */
+ copied_query = copyObject(dataQuery);
+ AcquireRewriteLocks(copied_query, true, false);
+ rewritten = QueryRewrite(copied_query);
+
+ /* SELECT should never rewrite to more or less than one SELECT query */
+ if (list_length(rewritten) != 1)
+ elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
+ dataQuery = (Query *) linitial(rewritten);
+
+ return dataQuery;
+}
--
2.25.1
v3-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchapplication/x-patch; name=v3-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchDownload
From c627d596cc4ac06e406952ed97b3e417b857aedd Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Fri, 8 Jan 2021 14:15:46 +0530
Subject: [PATCH v3 2/2] EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
Currently, explain/explain analyze refresh materialized view(RMV)
is not allowed. We do plan the materialized view query before every
refresh. I propose to show the explain/explain analyze of the
select part of the materialized view. It will be useful for the
user to know what exactly is being planned and executed as part of
RMV. Please note that we already have explain/explain analyze
CTAS/Create Mat View(CMV), where we show the explain/explain analyze
of the select part. This proposal will do the same thing.
The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.
---
src/backend/commands/explain.c | 50 +++++++++++-----
src/backend/commands/matview.c | 36 +++++++++---
src/backend/commands/prepare.c | 3 +-
src/backend/tcop/utility.c | 3 +-
src/include/commands/explain.h | 30 +++++++++-
src/include/commands/matview.h | 8 ++-
src/test/regress/expected/matview.out | 83 +++++++++++++++++++++++++++
src/test/regress/sql/matview.sql | 27 +++++++++
8 files changed, 215 insertions(+), 25 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5d7eb3574c..7343a916b0 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -17,6 +17,7 @@
#include "catalog/pg_type.h"
#include "commands/createas.h"
#include "commands/defrem.h"
+#include "commands/matview.h"
#include "commands/prepare.h"
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
@@ -53,10 +54,6 @@ explain_get_index_name_hook_type explain_get_index_name_hook = NULL;
#define X_CLOSE_IMMEDIATE 2
#define X_NOWHITESPACE 4
-static void ExplainOneQuery(Query *query, int cursorOptions,
- IntoClause *into, ExplainState *es,
- const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv);
static void ExplainPrintJIT(ExplainState *es, int jit_flags,
JitInstrumentation *ji);
static void report_triggers(ResultRelInfo *rInfo, bool show_relname,
@@ -274,7 +271,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
{
ExplainOneQuery(lfirst_node(Query, l),
CURSOR_OPT_PARALLEL_OK, NULL, es,
- pstate->p_sourcetext, params, pstate->p_queryEnv);
+ pstate->p_sourcetext, params, pstate->p_queryEnv,
+ NULL);
/* Separate plans with an appropriate separator */
if (lnext(rewritten, l) != NULL)
@@ -357,11 +355,11 @@ ExplainResultDesc(ExplainStmt *stmt)
*
* "into" is NULL unless we are explaining the contents of a CreateTableAsStmt.
*/
-static void
+void
ExplainOneQuery(Query *query, int cursorOptions,
IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv)
+ QueryEnvironment *queryEnv, RefreshMatViewInfo *matviewInfo)
{
/* planner will not cope with utility statements */
if (query->commandType == CMD_UTILITY)
@@ -402,7 +400,8 @@ ExplainOneQuery(Query *query, int cursorOptions,
/* run it (if needed) and produce output */
ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ matviewInfo);
}
}
@@ -455,7 +454,7 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
CURSOR_OPT_PARALLEL_OK, ctas->into, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv, NULL);
}
else if (IsA(utilityStmt, DeclareCursorStmt))
{
@@ -474,7 +473,8 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
dcs->options, NULL, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv,
+ NULL);
}
else if (IsA(utilityStmt, ExecuteStmt))
ExplainExecuteQuery((ExecuteStmt *) utilityStmt, into, es,
@@ -486,6 +486,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
else
ExplainDummyGroup("Notify", NULL, es);
}
+ else if(IsA(utilityStmt, RefreshMatViewStmt))
+ {
+ RefreshMatViewExplainInfo explainInfo;
+
+ explainInfo.es = es;
+ explainInfo.queryEnv = queryEnv;
+
+ ExecRefreshMatView((RefreshMatViewStmt *) utilityStmt,
+ queryString, params, NULL, &explainInfo);
+ }
else
{
if (es->format == EXPLAIN_FORMAT_TEXT)
@@ -512,7 +522,7 @@ void
ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
- const BufferUsage *bufusage)
+ const BufferUsage *bufusage, RefreshMatViewInfo *matviewInfo)
{
DestReceiver *dest;
QueryDesc *queryDesc;
@@ -553,6 +563,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
if (into)
dest = CreateIntoRelDestReceiver(into);
+ else if (matviewInfo && OidIsValid(matviewInfo->OIDNewHeap))
+ dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);
else
dest = None_Receiver;
@@ -577,8 +589,12 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
{
ScanDirection dir;
- /* EXPLAIN ANALYZE CREATE TABLE AS WITH NO DATA is weird */
- if (into && into->skipData)
+ /*
+ * EXPLAIN ANALYZE CREATE TABLE AS or REFRESH MATERIALIZED VIEW
+ * WITH NO DATA is weird.
+ */
+ if ((into && into->skipData) ||
+ (matviewInfo && matviewInfo->skipData))
dir = NoMovementScanDirection;
else
dir = ForwardScanDirection;
@@ -586,6 +602,14 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
/* run the plan */
ExecutorRun(queryDesc, dir, 0L, true);
+ /*
+ * Collect the number of rows inserted in case of REFRESH MATERIALIZED
+ * VIEW which will be used while merging the newly generated data with
+ * the existing materialized view data in ExecRefreshMatView.
+ */
+ if (matviewInfo)
+ matviewInfo->processed = queryDesc->estate->es_processed;
+
/* run cleanup too */
ExecutorFinish(queryDesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 06bd5629a8..66b8c4b402 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -146,7 +146,8 @@ SetMatViewPopulatedState(Relation relation, bool newstate)
*/
ObjectAddress
ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc)
+ ParamListInfo params, QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo)
{
Oid matviewOid;
Relation matviewRel;
@@ -182,8 +183,11 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
- &relpersistence);
+ if (explainInfo && !explainInfo->es->analyze)
+ OIDNewHeap = InvalidOid;
+ else
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -192,17 +196,35 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_RESTRICTED_OPERATION);
/* Generate the data, if wanted. */
- if (!stmt->skipData)
+ if (!stmt->skipData && !explainInfo)
{
dataQuery = rewrite_refresh_matview_query(dataQuery);
processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
queryString);
}
+ else if (explainInfo)
+ {
+ RefreshMatViewInfo matViewInfo;
+
+ matViewInfo.OIDNewHeap = OIDNewHeap;
+ matViewInfo.skipData = stmt->skipData;
+ matViewInfo.processed = 0;
+
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
+
+ ExplainOneQuery(dataQuery,
+ CURSOR_OPT_PARALLEL_OK, NULL, explainInfo->es,
+ queryString, params, explainInfo->queryEnv,
+ &matViewInfo);
+
+ processed = matViewInfo.processed;
+ }
- match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
- relowner, save_sec_context, relpersistence,
- processed);
+ if (OidIsValid(OIDNewHeap))
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 653ef8e41a..696d3343d4 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -672,7 +672,8 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
if (pstmt->commandType != CMD_UTILITY)
ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ NULL);
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 53a511f1da..ae7cef9c24 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1659,7 +1659,8 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
address = ExecRefreshMatView((RefreshMatViewStmt *) parsetree,
- queryString, params, qc);
+ queryString, params, qc,
+ NULL);
}
PG_FINALLY();
{
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7f27d92df1 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -61,6 +61,27 @@ typedef struct ExplainState
ExplainWorkersState *workers_state; /* needed if parallel plan */
} ExplainState;
+/*
+ * Refresh Materialized View information passed across functions for EXPLAIN
+ * execution.
+ */
+typedef struct RefreshMatViewInfo
+{
+ /* Oid of the new heap created. */
+ Oid OIDNewHeap;
+ /* Is WITH NO DATA clause specified? */
+ bool skipData;
+ /* Number of rows inserted. */
+ uint64 processed;
+} RefreshMatViewInfo;
+
+/* EXPLAIN information shared to ExecRefreshMatView(). */
+typedef struct RefreshMatViewExplainInfo
+{
+ ExplainState *es;
+ QueryEnvironment *queryEnv;
+} RefreshMatViewExplainInfo;
+
/* Hook for plugins to get control in ExplainOneQuery() */
typedef void (*ExplainOneQuery_hook_type) (Query *query,
int cursorOptions,
@@ -91,7 +112,14 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
- const BufferUsage *bufusage);
+ const BufferUsage *bufusage,
+ RefreshMatViewInfo *matviewInfo);
+
+extern void ExplainOneQuery(Query *query, int cursorOptions,
+ IntoClause *into, ExplainState *es,
+ const char *queryString, ParamListInfo params,
+ QueryEnvironment *queryEnv,
+ RefreshMatViewInfo *matviewInfo);
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
diff --git a/src/include/commands/matview.h b/src/include/commands/matview.h
index 214b1c1df6..1d60180ebc 100644
--- a/src/include/commands/matview.h
+++ b/src/include/commands/matview.h
@@ -15,6 +15,7 @@
#define MATVIEW_H
#include "catalog/objectaddress.h"
+#include "commands/explain.h"
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "tcop/dest.h"
@@ -23,8 +24,11 @@
extern void SetMatViewPopulatedState(Relation relation, bool newstate);
-extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc);
+extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt,
+ const char *queryString,
+ ParamListInfo params,
+ QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo);
extern DestReceiver *CreateTransientRelDestReceiver(Oid oid);
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..305e511fea 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -668,3 +668,86 @@ NOTICE: relation "matview_ine_tab" already exists, skipping
(0 rows)
DROP MATERIALIZED VIEW matview_ine_tab;
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+-----------------------------------------
+ Seq Scan on mv_exp_tbl (never executed)
+ Filter: (a > 5)
+(2 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+ERROR: materialized view "mv_exp" has not been populated
+HINT: Use the REFRESH MATERIALIZED VIEW command.
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
diff --git a/src/test/regress/sql/matview.sql b/src/test/regress/sql/matview.sql
index 4a4bd0d6b6..67a45bbde7 100644
--- a/src/test/regress/sql/matview.sql
+++ b/src/test/regress/sql/matview.sql
@@ -287,3 +287,30 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
CREATE MATERIALIZED VIEW IF NOT EXISTS matview_ine_tab AS
SELECT 1 / 0 WITH NO DATA; -- ok
DROP MATERIALIZED VIEW matview_ine_tab;
+
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
--
2.25.1
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Fri, 08 Jan 2021 at 17:24, Bharath Rupireddy wrote:
On Fri, Jan 8, 2021 at 1:50 PM japin <japinli@hotmail.com> wrote:
Thanks for updating the patch!
+ /* Get the data generating query. */ + dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);- /* - * Check for active uses of the relation in the current transaction, such - * as open scans. - * - * NB: We count on this to protect us against problems with refreshing the - * data using TABLE_INSERT_FROZEN. - */ - CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW"); + relowner = matviewRel->rd_rel->relowner;After apply the patch, there is a duplicate
relowner = matviewRel->rd_rel->relowner;
Corrected that.
+ else if(matviewInfo) + dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);If the `matviewInfo->OIDNewHeap` is invalid, IMO we don't need create
DestReceiver, isn't it? And we should add a space after `if`.Yes, we can skip creating the dest receiver when OIDNewHeap is
invalid, this can happen for plain explain refresh mat view case.if (explainInfo && !explainInfo->es->analyze)
OIDNewHeap = InvalidOid;
else
OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
&relpersistence);Since we don't call ExecutorRun for plain explain, we can skip the
dest receiver creation. I modified the code as below in explain.c.if (into)
dest = CreateIntoRelDestReceiver(into);
else if (matviewInfo && OidIsValid(matviewInfo->OIDNewHeap))
dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);
else
dest = None_Receiver;Thanks for taking a look at the patches.
Thanks!
Attaching v3 patches, please consider these for further review.
I find that both the declaration and definition of match_matview_with_new_data()
have a tab between type and variable. We can use pgindent to fix it.
What do you think?
static void
match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
^
Oid matviewOid, Oid OIDNewHeap, Oid relowner,
int save_sec_context, char relpersistence,
uint64 processed)
^
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Fri, Jan 8, 2021 at 9:50 PM japin <japinli@hotmail.com> wrote:
Attaching v3 patches, please consider these for further review.
I find that both the declaration and definition of match_matview_with_new_data()
have a tab between type and variable. We can use pgindent to fix it.
What do you think?static void
match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
^
Oid matviewOid, Oid OIDNewHeap, Oid relowner,
int save_sec_context, char relpersistence,
uint64 processed)
^
I ran pgindent on 0001 patch to fix the above. 0002 patch has no
changes. If I'm correct, pgindent will be run periodically on master.
Attaching v4 patch set for further review.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v4-0001-Rearrange-Refresh-Mat-View-Code.patchapplication/octet-stream; name=v4-0001-Rearrange-Refresh-Mat-View-Code.patchDownload
From dcdd85209f6fc9153f300ba659d92d27a03b69b8 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Sat, 9 Jan 2021 06:53:09 +0530
Subject: [PATCH v4 1/2] Rearrange Refresh Mat View Code
Currently, the function ExecRefreshMatView in matview.c is having
many lines of code which is not at all good from readability and
maintainability perspectives. This patch adds few functions and
moves the code from ExecRefreshMatView to them making the code
look better.
---
src/backend/commands/matview.c | 452 ++++++++++++++++++++-------------
1 file changed, 272 insertions(+), 180 deletions(-)
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..a8d65bba50 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -64,7 +64,7 @@ static void transientrel_startup(DestReceiver *self, int operation, TupleDesc ty
static bool transientrel_receive(TupleTableSlot *slot, DestReceiver *self);
static void transientrel_shutdown(DestReceiver *self);
static void transientrel_destroy(DestReceiver *self);
-static uint64 refresh_matview_datafill(DestReceiver *dest, Query *query,
+static uint64 refresh_matview_datafill(Oid OIDNewHeap, Query *query,
const char *queryString);
static char *make_temptable_name_n(char *tempname, int n);
static void refresh_by_match_merge(Oid matviewOid, Oid tempOid, Oid relowner,
@@ -73,6 +73,16 @@ static void refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersist
static bool is_usable_unique_index(Relation indexRel);
static void OpenMatViewIncrementalMaintenance(void);
static void CloseMatViewIncrementalMaintenance(void);
+static Query *get_matview_query(RefreshMatViewStmt *stmt, Relation *rel,
+ Oid *objectId);
+static Query *rewrite_refresh_matview_query(Query *dataQuery);
+static Oid get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, char *relpersistence);
+static void match_matview_with_new_data(RefreshMatViewStmt *stmt,
+ Relation matviewRel, Oid matviewOid,
+ Oid OIDNewHeap, Oid relowner,
+ int save_sec_context,
+ char relpersistence, uint64 processed);
/*
* SetMatViewPopulatedState
@@ -140,127 +150,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
{
Oid matviewOid;
Relation matviewRel;
- RewriteRule *rule;
- List *actions;
Query *dataQuery;
- Oid tableSpace;
- Oid relowner;
Oid OIDNewHeap;
- DestReceiver *dest;
uint64 processed = 0;
- bool concurrent;
- LOCKMODE lockmode;
+ Oid relowner;
char relpersistence;
Oid save_userid;
int save_sec_context;
int save_nestlevel;
ObjectAddress address;
- /* Determine strength of lock needed. */
- concurrent = stmt->concurrent;
- lockmode = concurrent ? ExclusiveLock : AccessExclusiveLock;
-
- /*
- * Get a lock until end of transaction.
- */
- matviewOid = RangeVarGetRelidExtended(stmt->relation,
- lockmode, 0,
- RangeVarCallbackOwnsTable, NULL);
- matviewRel = table_open(matviewOid, NoLock);
-
- /* Make sure it is a materialized view. */
- if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("\"%s\" is not a materialized view",
- RelationGetRelationName(matviewRel))));
-
- /* Check that CONCURRENTLY is not specified if not populated. */
- if (concurrent && !RelationIsPopulated(matviewRel))
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
-
- /* Check that conflicting options have not been specified. */
- if (concurrent && stmt->skipData)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
-
- /*
- * Check that everything is correct for a refresh. Problems at this point
- * are internal errors, so elog is sufficient.
- */
- if (matviewRel->rd_rel->relhasrules == false ||
- matviewRel->rd_rules->numLocks < 1)
- elog(ERROR,
- "materialized view \"%s\" is missing rewrite information",
- RelationGetRelationName(matviewRel));
-
- if (matviewRel->rd_rules->numLocks > 1)
- elog(ERROR,
- "materialized view \"%s\" has too many rules",
- RelationGetRelationName(matviewRel));
-
- rule = matviewRel->rd_rules->rules[0];
- if (rule->event != CMD_SELECT || !(rule->isInstead))
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
- RelationGetRelationName(matviewRel));
-
- actions = rule->actions;
- if (list_length(actions) != 1)
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a single action",
- RelationGetRelationName(matviewRel));
-
- /*
- * Check that there is a unique index with no WHERE clause on one or more
- * columns of the materialized view if CONCURRENTLY is specified.
- */
- if (concurrent)
- {
- List *indexoidlist = RelationGetIndexList(matviewRel);
- ListCell *indexoidscan;
- bool hasUniqueIndex = false;
-
- foreach(indexoidscan, indexoidlist)
- {
- Oid indexoid = lfirst_oid(indexoidscan);
- Relation indexRel;
-
- indexRel = index_open(indexoid, AccessShareLock);
- hasUniqueIndex = is_usable_unique_index(indexRel);
- index_close(indexRel, AccessShareLock);
- if (hasUniqueIndex)
- break;
- }
-
- list_free(indexoidlist);
-
- if (!hasUniqueIndex)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot refresh materialized view \"%s\" concurrently",
- quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
- RelationGetRelationName(matviewRel))),
- errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
- }
-
- /*
- * The stored query was rewritten at the time of the MV definition, but
- * has not been scribbled on by the planner.
- */
- dataQuery = linitial_node(Query, actions);
-
- /*
- * Check for active uses of the relation in the current transaction, such
- * as open scans.
- *
- * NB: We count on this to protect us against problems with refreshing the
- * data using TABLE_INSERT_FROZEN.
- */
- CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+ /* Get the data generating query. */
+ dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);
/*
* Tentatively mark the matview as populated or not (this will roll back
@@ -281,27 +182,8 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- /* Concurrent refresh builds new data in temp tablespace, and does diff. */
- if (concurrent)
- {
- tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
- relpersistence = RELPERSISTENCE_TEMP;
- }
- else
- {
- tableSpace = matviewRel->rd_rel->reltablespace;
- relpersistence = matviewRel->rd_rel->relpersistence;
- }
-
- /*
- * Create the transient table that will receive the regenerated data. Lock
- * it against access by any other process until commit (by which time it
- * will be gone).
- */
- OIDNewHeap = make_new_heap(matviewOid, tableSpace, relpersistence,
- ExclusiveLock);
- LockRelationOid(OIDNewHeap, AccessExclusiveLock);
- dest = CreateTransientRelDestReceiver(OIDNewHeap);
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -311,40 +193,16 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/* Generate the data, if wanted. */
if (!stmt->skipData)
- processed = refresh_matview_datafill(dest, dataQuery, queryString);
-
- /* Make the matview match the newly generated data. */
- if (concurrent)
{
- int old_depth = matview_maintenance_depth;
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
- PG_TRY();
- {
- refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
- save_sec_context);
- }
- PG_CATCH();
- {
- matview_maintenance_depth = old_depth;
- PG_RE_THROW();
- }
- PG_END_TRY();
- Assert(matview_maintenance_depth == old_depth);
+ processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
+ queryString);
}
- else
- {
- refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
- /*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
- */
- pgstat_count_truncate(matviewRel);
- if (!stmt->skipData)
- pgstat_count_heap_insert(matviewRel, processed);
- }
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
@@ -373,30 +231,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/*
* refresh_matview_datafill
*
- * Execute the given query, sending result rows to "dest" (which will
- * insert them into the target matview).
+ * Create dest receiver and execute the given query, sending result rows to the
+ * dest receiver which will insert them into the target materialized view.
*
* Returns number of rows inserted.
*/
static uint64
-refresh_matview_datafill(DestReceiver *dest, Query *query,
- const char *queryString)
+refresh_matview_datafill(Oid OIDNewHeap, Query *query, const char *queryString)
{
- List *rewritten;
PlannedStmt *plan;
QueryDesc *queryDesc;
- Query *copied_query;
uint64 processed;
-
- /* Lock and rewrite, using a copy to preserve the original query. */
- copied_query = copyObject(query);
- AcquireRewriteLocks(copied_query, true, false);
- rewritten = QueryRewrite(copied_query);
-
- /* SELECT should never rewrite to more or less than one SELECT query */
- if (list_length(rewritten) != 1)
- elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
- query = (Query *) linitial(rewritten);
+ DestReceiver *dest;
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
@@ -413,6 +259,8 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
PushCopiedSnapshot(GetActiveSnapshot());
UpdateActiveSnapshotCommandId();
+ dest = CreateTransientRelDestReceiver(OIDNewHeap);
+
/* Create a QueryDesc, redirecting output to our tuple receiver */
queryDesc = CreateQueryDesc(plan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
@@ -933,3 +781,247 @@ CloseMatViewIncrementalMaintenance(void)
matview_maintenance_depth--;
Assert(matview_maintenance_depth >= 0);
}
+
+/*
+ * get_matview_query
+ *
+ * Open the refresh materialized view relation, perform sanity checks and also
+ * get the associated data generating query from it.
+ *
+ * Note that the refresh materialized view relation is opened here, it has to
+ * be closed in the caller.
+ */
+static Query *
+get_matview_query(RefreshMatViewStmt *stmt, Relation *rel, Oid *objectId)
+{
+ Oid matviewOid;
+ Relation matviewRel;
+ RewriteRule *rule;
+ List *actions;
+ Query *dataQuery;
+ LOCKMODE lockmode;
+
+ /* Determine strength of lock needed. */
+ lockmode = stmt->concurrent ? ExclusiveLock : AccessExclusiveLock;
+
+ /* Get a lock until end of transaction. */
+ matviewOid = RangeVarGetRelidExtended(stmt->relation, lockmode, 0,
+ RangeVarCallbackOwnsTable, NULL);
+ matviewRel = table_open(matviewOid, NoLock);
+
+ /* Make sure it is a materialized view. */
+ if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("\"%s\" is not a materialized view",
+ RelationGetRelationName(matviewRel))));
+
+ /* Check that CONCURRENTLY is not specified if not populated. */
+ if (stmt->concurrent && !RelationIsPopulated(matviewRel))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
+
+ /* Check that conflicting options have not been specified. */
+ if (stmt->concurrent && stmt->skipData)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
+
+ /*
+ * Check that everything is correct for a refresh. Problems at this point
+ * are internal errors, so elog is sufficient.
+ */
+ if (matviewRel->rd_rel->relhasrules == false ||
+ matviewRel->rd_rules->numLocks < 1)
+ elog(ERROR,
+ "materialized view \"%s\" is missing rewrite information",
+ RelationGetRelationName(matviewRel));
+
+ if (matviewRel->rd_rules->numLocks > 1)
+ elog(ERROR,
+ "materialized view \"%s\" has too many rules",
+ RelationGetRelationName(matviewRel));
+
+ rule = matviewRel->rd_rules->rules[0];
+ if (rule->event != CMD_SELECT || !(rule->isInstead))
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
+ RelationGetRelationName(matviewRel));
+
+ actions = rule->actions;
+ if (list_length(actions) != 1)
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a single action",
+ RelationGetRelationName(matviewRel));
+
+ /*
+ * Check that there is a unique index with no WHERE clause on one or more
+ * columns of the materialized view if CONCURRENTLY is specified.
+ */
+ if (stmt->concurrent)
+ {
+ List *indexoidlist = RelationGetIndexList(matviewRel);
+ ListCell *indexoidscan;
+ bool hasUniqueIndex = false;
+
+ foreach(indexoidscan, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(indexoidscan);
+ Relation indexRel;
+
+ indexRel = index_open(indexoid, AccessShareLock);
+ hasUniqueIndex = is_usable_unique_index(indexRel);
+ index_close(indexRel, AccessShareLock);
+ if (hasUniqueIndex)
+ break;
+ }
+
+ list_free(indexoidlist);
+
+ if (!hasUniqueIndex)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot refresh materialized view \"%s\" concurrently",
+ quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
+ RelationGetRelationName(matviewRel))),
+ errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
+ }
+
+ /*
+ * The stored query was rewritten at the time of the MV definition, but
+ * has not been scribbled on by the planner.
+ */
+ dataQuery = linitial_node(Query, actions);
+
+ /*
+ * Check for active uses of the relation in the current transaction, such
+ * as open scans.
+ *
+ * NB: We count on this to protect us against problems with refreshing the
+ * data using TABLE_INSERT_FROZEN.
+ */
+ CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+
+ *rel = matviewRel;
+ *objectId = matviewOid;
+
+ return dataQuery;
+}
+
+/*
+ * get_new_heap_oid
+ *
+ * Create a new heap and return its oid to which the refresh materialized view
+ * data is inserted into.
+ */
+static Oid
+get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid matviewOid,
+ char *relpersistence)
+{
+ Oid OIDNewHeap;
+ bool concurrent;
+ Oid tableSpace;
+
+ concurrent = stmt->concurrent;
+
+ /* Concurrent refresh builds new data in temp tablespace, and does diff. */
+ if (concurrent)
+ {
+ tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
+ *relpersistence = RELPERSISTENCE_TEMP;
+ }
+ else
+ {
+ tableSpace = matviewRel->rd_rel->reltablespace;
+ *relpersistence = matviewRel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Create the transient table that will receive the regenerated data. Lock
+ * it against access by any other process until commit (by which time it
+ * will be gone).
+ */
+ OIDNewHeap = make_new_heap(matviewOid, tableSpace, *relpersistence,
+ ExclusiveLock);
+ LockRelationOid(OIDNewHeap, AccessExclusiveLock);
+
+ return OIDNewHeap;
+}
+
+/*
+ * match_matview_with_new_data
+ *
+ * Arrange the materialized view newly generated data to match the existing
+ * data i.e merge in case of CONCURRENTLY otherwise perform heap swap and
+ * truncate the materialized view.
+ */
+static void
+match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, Oid OIDNewHeap, Oid relowner,
+ int save_sec_context, char relpersistence,
+ uint64 processed)
+{
+ /* Make the materialized view match the newly generated data. */
+ if (stmt->concurrent)
+ {
+ int old_depth = matview_maintenance_depth;
+
+ PG_TRY();
+ {
+ refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
+ save_sec_context);
+ }
+ PG_CATCH();
+ {
+ matview_maintenance_depth = old_depth;
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ Assert(matview_maintenance_depth == old_depth);
+ }
+ else
+ {
+ refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
+
+ /*
+ * Inform stats collector about our activity: basically, we truncated
+ * the materialized view and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the
+ * inserts and deletes it issues get counted by lower-level code.)
+ */
+ pgstat_count_truncate(matviewRel);
+ if (!stmt->skipData)
+ pgstat_count_heap_insert(matviewRel, processed);
+ }
+}
+
+/*
+ * rewrite_refresh_matview_query
+ *
+ * Rewrite the refresh materialized view data generating query.
+ *
+ * Work on the copied query to preserve the original query. Because the
+ * rewriter and planner tend to scribble on the input, we make a preliminary
+ * copy of the source querytree. This prevents problems in the case that
+ * REFRESH MATERIALIZED VIEW is in a portal or plpgsql function and is executed
+ * repeatedly. (See also the same hack in EXPLAIN and PREPARE.)
+ */
+static Query *
+rewrite_refresh_matview_query(Query *dataQuery)
+{
+ List *rewritten;
+ Query *copied_query;
+
+ /* Lock and rewrite, using a copy to preserve the original query. */
+ copied_query = copyObject(dataQuery);
+ AcquireRewriteLocks(copied_query, true, false);
+ rewritten = QueryRewrite(copied_query);
+
+ /* SELECT should never rewrite to more or less than one SELECT query */
+ if (list_length(rewritten) != 1)
+ elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
+ dataQuery = (Query *) linitial(rewritten);
+
+ return dataQuery;
+}
--
2.25.1
v4-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchapplication/octet-stream; name=v4-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchDownload
From c627d596cc4ac06e406952ed97b3e417b857aedd Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Fri, 8 Jan 2021 14:15:46 +0530
Subject: [PATCH v4 2/2] EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
Currently, explain/explain analyze refresh materialized view(RMV)
is not allowed. We do plan the materialized view query before every
refresh. I propose to show the explain/explain analyze of the
select part of the materialized view. It will be useful for the
user to know what exactly is being planned and executed as part of
RMV. Please note that we already have explain/explain analyze
CTAS/Create Mat View(CMV), where we show the explain/explain analyze
of the select part. This proposal will do the same thing.
The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.
---
src/backend/commands/explain.c | 50 +++++++++++-----
src/backend/commands/matview.c | 36 +++++++++---
src/backend/commands/prepare.c | 3 +-
src/backend/tcop/utility.c | 3 +-
src/include/commands/explain.h | 30 +++++++++-
src/include/commands/matview.h | 8 ++-
src/test/regress/expected/matview.out | 83 +++++++++++++++++++++++++++
src/test/regress/sql/matview.sql | 27 +++++++++
8 files changed, 215 insertions(+), 25 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5d7eb3574c..7343a916b0 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -17,6 +17,7 @@
#include "catalog/pg_type.h"
#include "commands/createas.h"
#include "commands/defrem.h"
+#include "commands/matview.h"
#include "commands/prepare.h"
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
@@ -53,10 +54,6 @@ explain_get_index_name_hook_type explain_get_index_name_hook = NULL;
#define X_CLOSE_IMMEDIATE 2
#define X_NOWHITESPACE 4
-static void ExplainOneQuery(Query *query, int cursorOptions,
- IntoClause *into, ExplainState *es,
- const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv);
static void ExplainPrintJIT(ExplainState *es, int jit_flags,
JitInstrumentation *ji);
static void report_triggers(ResultRelInfo *rInfo, bool show_relname,
@@ -274,7 +271,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
{
ExplainOneQuery(lfirst_node(Query, l),
CURSOR_OPT_PARALLEL_OK, NULL, es,
- pstate->p_sourcetext, params, pstate->p_queryEnv);
+ pstate->p_sourcetext, params, pstate->p_queryEnv,
+ NULL);
/* Separate plans with an appropriate separator */
if (lnext(rewritten, l) != NULL)
@@ -357,11 +355,11 @@ ExplainResultDesc(ExplainStmt *stmt)
*
* "into" is NULL unless we are explaining the contents of a CreateTableAsStmt.
*/
-static void
+void
ExplainOneQuery(Query *query, int cursorOptions,
IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv)
+ QueryEnvironment *queryEnv, RefreshMatViewInfo *matviewInfo)
{
/* planner will not cope with utility statements */
if (query->commandType == CMD_UTILITY)
@@ -402,7 +400,8 @@ ExplainOneQuery(Query *query, int cursorOptions,
/* run it (if needed) and produce output */
ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ matviewInfo);
}
}
@@ -455,7 +454,7 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
CURSOR_OPT_PARALLEL_OK, ctas->into, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv, NULL);
}
else if (IsA(utilityStmt, DeclareCursorStmt))
{
@@ -474,7 +473,8 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
dcs->options, NULL, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv,
+ NULL);
}
else if (IsA(utilityStmt, ExecuteStmt))
ExplainExecuteQuery((ExecuteStmt *) utilityStmt, into, es,
@@ -486,6 +486,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
else
ExplainDummyGroup("Notify", NULL, es);
}
+ else if(IsA(utilityStmt, RefreshMatViewStmt))
+ {
+ RefreshMatViewExplainInfo explainInfo;
+
+ explainInfo.es = es;
+ explainInfo.queryEnv = queryEnv;
+
+ ExecRefreshMatView((RefreshMatViewStmt *) utilityStmt,
+ queryString, params, NULL, &explainInfo);
+ }
else
{
if (es->format == EXPLAIN_FORMAT_TEXT)
@@ -512,7 +522,7 @@ void
ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
- const BufferUsage *bufusage)
+ const BufferUsage *bufusage, RefreshMatViewInfo *matviewInfo)
{
DestReceiver *dest;
QueryDesc *queryDesc;
@@ -553,6 +563,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
if (into)
dest = CreateIntoRelDestReceiver(into);
+ else if (matviewInfo && OidIsValid(matviewInfo->OIDNewHeap))
+ dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);
else
dest = None_Receiver;
@@ -577,8 +589,12 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
{
ScanDirection dir;
- /* EXPLAIN ANALYZE CREATE TABLE AS WITH NO DATA is weird */
- if (into && into->skipData)
+ /*
+ * EXPLAIN ANALYZE CREATE TABLE AS or REFRESH MATERIALIZED VIEW
+ * WITH NO DATA is weird.
+ */
+ if ((into && into->skipData) ||
+ (matviewInfo && matviewInfo->skipData))
dir = NoMovementScanDirection;
else
dir = ForwardScanDirection;
@@ -586,6 +602,14 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
/* run the plan */
ExecutorRun(queryDesc, dir, 0L, true);
+ /*
+ * Collect the number of rows inserted in case of REFRESH MATERIALIZED
+ * VIEW which will be used while merging the newly generated data with
+ * the existing materialized view data in ExecRefreshMatView.
+ */
+ if (matviewInfo)
+ matviewInfo->processed = queryDesc->estate->es_processed;
+
/* run cleanup too */
ExecutorFinish(queryDesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 06bd5629a8..66b8c4b402 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -146,7 +146,8 @@ SetMatViewPopulatedState(Relation relation, bool newstate)
*/
ObjectAddress
ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc)
+ ParamListInfo params, QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo)
{
Oid matviewOid;
Relation matviewRel;
@@ -182,8 +183,11 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
- &relpersistence);
+ if (explainInfo && !explainInfo->es->analyze)
+ OIDNewHeap = InvalidOid;
+ else
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -192,17 +196,35 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_RESTRICTED_OPERATION);
/* Generate the data, if wanted. */
- if (!stmt->skipData)
+ if (!stmt->skipData && !explainInfo)
{
dataQuery = rewrite_refresh_matview_query(dataQuery);
processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
queryString);
}
+ else if (explainInfo)
+ {
+ RefreshMatViewInfo matViewInfo;
+
+ matViewInfo.OIDNewHeap = OIDNewHeap;
+ matViewInfo.skipData = stmt->skipData;
+ matViewInfo.processed = 0;
+
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
+
+ ExplainOneQuery(dataQuery,
+ CURSOR_OPT_PARALLEL_OK, NULL, explainInfo->es,
+ queryString, params, explainInfo->queryEnv,
+ &matViewInfo);
+
+ processed = matViewInfo.processed;
+ }
- match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
- relowner, save_sec_context, relpersistence,
- processed);
+ if (OidIsValid(OIDNewHeap))
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 653ef8e41a..696d3343d4 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -672,7 +672,8 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
if (pstmt->commandType != CMD_UTILITY)
ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ NULL);
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 53a511f1da..ae7cef9c24 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1659,7 +1659,8 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
address = ExecRefreshMatView((RefreshMatViewStmt *) parsetree,
- queryString, params, qc);
+ queryString, params, qc,
+ NULL);
}
PG_FINALLY();
{
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7f27d92df1 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -61,6 +61,27 @@ typedef struct ExplainState
ExplainWorkersState *workers_state; /* needed if parallel plan */
} ExplainState;
+/*
+ * Refresh Materialized View information passed across functions for EXPLAIN
+ * execution.
+ */
+typedef struct RefreshMatViewInfo
+{
+ /* Oid of the new heap created. */
+ Oid OIDNewHeap;
+ /* Is WITH NO DATA clause specified? */
+ bool skipData;
+ /* Number of rows inserted. */
+ uint64 processed;
+} RefreshMatViewInfo;
+
+/* EXPLAIN information shared to ExecRefreshMatView(). */
+typedef struct RefreshMatViewExplainInfo
+{
+ ExplainState *es;
+ QueryEnvironment *queryEnv;
+} RefreshMatViewExplainInfo;
+
/* Hook for plugins to get control in ExplainOneQuery() */
typedef void (*ExplainOneQuery_hook_type) (Query *query,
int cursorOptions,
@@ -91,7 +112,14 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
- const BufferUsage *bufusage);
+ const BufferUsage *bufusage,
+ RefreshMatViewInfo *matviewInfo);
+
+extern void ExplainOneQuery(Query *query, int cursorOptions,
+ IntoClause *into, ExplainState *es,
+ const char *queryString, ParamListInfo params,
+ QueryEnvironment *queryEnv,
+ RefreshMatViewInfo *matviewInfo);
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
diff --git a/src/include/commands/matview.h b/src/include/commands/matview.h
index 214b1c1df6..1d60180ebc 100644
--- a/src/include/commands/matview.h
+++ b/src/include/commands/matview.h
@@ -15,6 +15,7 @@
#define MATVIEW_H
#include "catalog/objectaddress.h"
+#include "commands/explain.h"
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "tcop/dest.h"
@@ -23,8 +24,11 @@
extern void SetMatViewPopulatedState(Relation relation, bool newstate);
-extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc);
+extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt,
+ const char *queryString,
+ ParamListInfo params,
+ QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo);
extern DestReceiver *CreateTransientRelDestReceiver(Oid oid);
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..305e511fea 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -668,3 +668,86 @@ NOTICE: relation "matview_ine_tab" already exists, skipping
(0 rows)
DROP MATERIALIZED VIEW matview_ine_tab;
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+-----------------------------------------
+ Seq Scan on mv_exp_tbl (never executed)
+ Filter: (a > 5)
+(2 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+ERROR: materialized view "mv_exp" has not been populated
+HINT: Use the REFRESH MATERIALIZED VIEW command.
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
diff --git a/src/test/regress/sql/matview.sql b/src/test/regress/sql/matview.sql
index 4a4bd0d6b6..67a45bbde7 100644
--- a/src/test/regress/sql/matview.sql
+++ b/src/test/regress/sql/matview.sql
@@ -287,3 +287,30 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
CREATE MATERIALIZED VIEW IF NOT EXISTS matview_ine_tab AS
SELECT 1 / 0 WITH NO DATA; -- ok
DROP MATERIALIZED VIEW matview_ine_tab;
+
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
--
2.25.1
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Sat, 09 Jan 2021 at 09:38, Bharath Rupireddy wrote:
On Fri, Jan 8, 2021 at 9:50 PM japin <japinli@hotmail.com> wrote:
Attaching v3 patches, please consider these for further review.
I find that both the declaration and definition of match_matview_with_new_data()
have a tab between type and variable. We can use pgindent to fix it.
What do you think?static void
match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
^
Oid matviewOid, Oid OIDNewHeap, Oid relowner,
int save_sec_context, char relpersistence,
uint64 processed)
^I ran pgindent on 0001 patch to fix the above. 0002 patch has no
changes. If I'm correct, pgindent will be run periodically on master.
Thanks for your point out. I don't know before.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
Hi Japin,
On 1/8/21 9:02 PM, japin wrote:
On Sat, 09 Jan 2021 at 09:38, Bharath Rupireddy wrote:
On Fri, Jan 8, 2021 at 9:50 PM japin <japinli@hotmail.com> wrote:
I ran pgindent on 0001 patch to fix the above. 0002 patch has no
changes. If I'm correct, pgindent will be run periodically on master.Thanks for your point out. I don't know before.
Do you know if you will have time to review this patch during the
current commitfest?
Regards,
--
-David
david@pgmasters.net
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Wed, 03 Mar 2021 at 20:56, David Steele <david@pgmasters.net> wrote:
Do you know if you will have time to review this patch during the
current commitfest?
Sorry for the late reply! I think I have time to review this patch
and I will do it later.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Thu, Mar 4, 2021 at 11:41 AM Japin Li <japinli@hotmail.com> wrote:
On Wed, 03 Mar 2021 at 20:56, David Steele <david@pgmasters.net> wrote:
Do you know if you will have time to review this patch during the
current commitfest?Sorry for the late reply! I think I have time to review this patch
and I will do it later.
Thanks! I will look forward for more review comments.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Thu, 04 Mar 2021 at 14:53, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
Thanks! I will look forward for more review comments.
v4-0001-Rearrange-Refresh-Mat-View-Code.patch
---------------------------------------------
+static Oid
+get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid matviewOid,
+ char *relpersistence)
+{
+ Oid OIDNewHeap;
+ bool concurrent;
+ Oid tableSpace;
+
+ concurrent = stmt->concurrent;
+
+ /* Concurrent refresh builds new data in temp tablespace, and does diff. */
+ if (concurrent)
+ {
+ tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
+ *relpersistence = RELPERSISTENCE_TEMP;
+ }Since the concurrent only use in one place, I think we can remove the local variable
concurrent in get_new_heap_oid().
The others looks good to me.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Fri, Mar 5, 2021 at 9:32 AM Japin Li <japinli@hotmail.com> wrote:
On Thu, 04 Mar 2021 at 14:53, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
Thanks! I will look forward for more review comments.
v4-0001-Rearrange-Refresh-Mat-View-Code.patch
---------------------------------------------+static Oid +get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid matviewOid, + char *relpersistence) +{ + Oid OIDNewHeap; + bool concurrent; + Oid tableSpace; + + concurrent = stmt->concurrent; + + /* Concurrent refresh builds new data in temp tablespace, and does diff. */ + if (concurrent) + { + tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false); + *relpersistence = RELPERSISTENCE_TEMP; + }Since the concurrent only use in one place, I think we can remove the local variable
concurrent in get_new_heap_oid().
Done.
The others looks good to me.
Thanks.
Attaching v5 patch set for further review.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v5-0001-Rearrange-Refresh-Mat-View-Code.patchapplication/x-patch; name=v5-0001-Rearrange-Refresh-Mat-View-Code.patchDownload
From 26970ffd33e67324609a03f0f61eeb6406216a7f Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Fri, 5 Mar 2021 15:47:12 +0530
Subject: [PATCH v5 1/2] Rearrange Refresh Mat View Code
Currently, the function ExecRefreshMatView in matview.c is having
many lines of code which is not at all good from readability and
maintainability perspectives. This patch adds few functions and
moves the code from ExecRefreshMatView to them making the code
look better.
---
src/backend/commands/matview.c | 449 ++++++++++++++++++++-------------
1 file changed, 269 insertions(+), 180 deletions(-)
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..18e18fa627 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -64,7 +64,7 @@ static void transientrel_startup(DestReceiver *self, int operation, TupleDesc ty
static bool transientrel_receive(TupleTableSlot *slot, DestReceiver *self);
static void transientrel_shutdown(DestReceiver *self);
static void transientrel_destroy(DestReceiver *self);
-static uint64 refresh_matview_datafill(DestReceiver *dest, Query *query,
+static uint64 refresh_matview_datafill(Oid OIDNewHeap, Query *query,
const char *queryString);
static char *make_temptable_name_n(char *tempname, int n);
static void refresh_by_match_merge(Oid matviewOid, Oid tempOid, Oid relowner,
@@ -73,6 +73,16 @@ static void refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersist
static bool is_usable_unique_index(Relation indexRel);
static void OpenMatViewIncrementalMaintenance(void);
static void CloseMatViewIncrementalMaintenance(void);
+static Query *get_matview_query(RefreshMatViewStmt *stmt, Relation *rel,
+ Oid *objectId);
+static Query *rewrite_refresh_matview_query(Query *dataQuery);
+static Oid get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, char *relpersistence);
+static void match_matview_with_new_data(RefreshMatViewStmt *stmt,
+ Relation matviewRel, Oid matviewOid,
+ Oid OIDNewHeap, Oid relowner,
+ int save_sec_context,
+ char relpersistence, uint64 processed);
/*
* SetMatViewPopulatedState
@@ -140,127 +150,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
{
Oid matviewOid;
Relation matviewRel;
- RewriteRule *rule;
- List *actions;
Query *dataQuery;
- Oid tableSpace;
- Oid relowner;
Oid OIDNewHeap;
- DestReceiver *dest;
uint64 processed = 0;
- bool concurrent;
- LOCKMODE lockmode;
+ Oid relowner;
char relpersistence;
Oid save_userid;
int save_sec_context;
int save_nestlevel;
ObjectAddress address;
- /* Determine strength of lock needed. */
- concurrent = stmt->concurrent;
- lockmode = concurrent ? ExclusiveLock : AccessExclusiveLock;
-
- /*
- * Get a lock until end of transaction.
- */
- matviewOid = RangeVarGetRelidExtended(stmt->relation,
- lockmode, 0,
- RangeVarCallbackOwnsTable, NULL);
- matviewRel = table_open(matviewOid, NoLock);
-
- /* Make sure it is a materialized view. */
- if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("\"%s\" is not a materialized view",
- RelationGetRelationName(matviewRel))));
-
- /* Check that CONCURRENTLY is not specified if not populated. */
- if (concurrent && !RelationIsPopulated(matviewRel))
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
-
- /* Check that conflicting options have not been specified. */
- if (concurrent && stmt->skipData)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
-
- /*
- * Check that everything is correct for a refresh. Problems at this point
- * are internal errors, so elog is sufficient.
- */
- if (matviewRel->rd_rel->relhasrules == false ||
- matviewRel->rd_rules->numLocks < 1)
- elog(ERROR,
- "materialized view \"%s\" is missing rewrite information",
- RelationGetRelationName(matviewRel));
-
- if (matviewRel->rd_rules->numLocks > 1)
- elog(ERROR,
- "materialized view \"%s\" has too many rules",
- RelationGetRelationName(matviewRel));
-
- rule = matviewRel->rd_rules->rules[0];
- if (rule->event != CMD_SELECT || !(rule->isInstead))
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
- RelationGetRelationName(matviewRel));
-
- actions = rule->actions;
- if (list_length(actions) != 1)
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a single action",
- RelationGetRelationName(matviewRel));
-
- /*
- * Check that there is a unique index with no WHERE clause on one or more
- * columns of the materialized view if CONCURRENTLY is specified.
- */
- if (concurrent)
- {
- List *indexoidlist = RelationGetIndexList(matviewRel);
- ListCell *indexoidscan;
- bool hasUniqueIndex = false;
-
- foreach(indexoidscan, indexoidlist)
- {
- Oid indexoid = lfirst_oid(indexoidscan);
- Relation indexRel;
-
- indexRel = index_open(indexoid, AccessShareLock);
- hasUniqueIndex = is_usable_unique_index(indexRel);
- index_close(indexRel, AccessShareLock);
- if (hasUniqueIndex)
- break;
- }
-
- list_free(indexoidlist);
-
- if (!hasUniqueIndex)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot refresh materialized view \"%s\" concurrently",
- quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
- RelationGetRelationName(matviewRel))),
- errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
- }
-
- /*
- * The stored query was rewritten at the time of the MV definition, but
- * has not been scribbled on by the planner.
- */
- dataQuery = linitial_node(Query, actions);
-
- /*
- * Check for active uses of the relation in the current transaction, such
- * as open scans.
- *
- * NB: We count on this to protect us against problems with refreshing the
- * data using TABLE_INSERT_FROZEN.
- */
- CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+ /* Get the data generating query. */
+ dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);
/*
* Tentatively mark the matview as populated or not (this will roll back
@@ -281,27 +182,8 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- /* Concurrent refresh builds new data in temp tablespace, and does diff. */
- if (concurrent)
- {
- tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
- relpersistence = RELPERSISTENCE_TEMP;
- }
- else
- {
- tableSpace = matviewRel->rd_rel->reltablespace;
- relpersistence = matviewRel->rd_rel->relpersistence;
- }
-
- /*
- * Create the transient table that will receive the regenerated data. Lock
- * it against access by any other process until commit (by which time it
- * will be gone).
- */
- OIDNewHeap = make_new_heap(matviewOid, tableSpace, relpersistence,
- ExclusiveLock);
- LockRelationOid(OIDNewHeap, AccessExclusiveLock);
- dest = CreateTransientRelDestReceiver(OIDNewHeap);
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -311,40 +193,16 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/* Generate the data, if wanted. */
if (!stmt->skipData)
- processed = refresh_matview_datafill(dest, dataQuery, queryString);
-
- /* Make the matview match the newly generated data. */
- if (concurrent)
{
- int old_depth = matview_maintenance_depth;
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
- PG_TRY();
- {
- refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
- save_sec_context);
- }
- PG_CATCH();
- {
- matview_maintenance_depth = old_depth;
- PG_RE_THROW();
- }
- PG_END_TRY();
- Assert(matview_maintenance_depth == old_depth);
+ processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
+ queryString);
}
- else
- {
- refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
- /*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
- */
- pgstat_count_truncate(matviewRel);
- if (!stmt->skipData)
- pgstat_count_heap_insert(matviewRel, processed);
- }
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
@@ -373,30 +231,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/*
* refresh_matview_datafill
*
- * Execute the given query, sending result rows to "dest" (which will
- * insert them into the target matview).
+ * Create dest receiver and execute the given query, sending result rows to the
+ * dest receiver which will insert them into the target materialized view.
*
* Returns number of rows inserted.
*/
static uint64
-refresh_matview_datafill(DestReceiver *dest, Query *query,
- const char *queryString)
+refresh_matview_datafill(Oid OIDNewHeap, Query *query, const char *queryString)
{
- List *rewritten;
PlannedStmt *plan;
QueryDesc *queryDesc;
- Query *copied_query;
uint64 processed;
-
- /* Lock and rewrite, using a copy to preserve the original query. */
- copied_query = copyObject(query);
- AcquireRewriteLocks(copied_query, true, false);
- rewritten = QueryRewrite(copied_query);
-
- /* SELECT should never rewrite to more or less than one SELECT query */
- if (list_length(rewritten) != 1)
- elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
- query = (Query *) linitial(rewritten);
+ DestReceiver *dest;
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
@@ -413,6 +259,8 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
PushCopiedSnapshot(GetActiveSnapshot());
UpdateActiveSnapshotCommandId();
+ dest = CreateTransientRelDestReceiver(OIDNewHeap);
+
/* Create a QueryDesc, redirecting output to our tuple receiver */
queryDesc = CreateQueryDesc(plan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
@@ -933,3 +781,244 @@ CloseMatViewIncrementalMaintenance(void)
matview_maintenance_depth--;
Assert(matview_maintenance_depth >= 0);
}
+
+/*
+ * get_matview_query
+ *
+ * Open the refresh materialized view relation, perform sanity checks and also
+ * get the associated data generating query from it.
+ *
+ * Note that the refresh materialized view relation is opened here, it has to
+ * be closed in the caller.
+ */
+static Query *
+get_matview_query(RefreshMatViewStmt *stmt, Relation *rel, Oid *objectId)
+{
+ Oid matviewOid;
+ Relation matviewRel;
+ RewriteRule *rule;
+ List *actions;
+ Query *dataQuery;
+ LOCKMODE lockmode;
+
+ /* Determine strength of lock needed. */
+ lockmode = stmt->concurrent ? ExclusiveLock : AccessExclusiveLock;
+
+ /* Get a lock until end of transaction. */
+ matviewOid = RangeVarGetRelidExtended(stmt->relation, lockmode, 0,
+ RangeVarCallbackOwnsTable, NULL);
+ matviewRel = table_open(matviewOid, NoLock);
+
+ /* Make sure it is a materialized view. */
+ if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("\"%s\" is not a materialized view",
+ RelationGetRelationName(matviewRel))));
+
+ /* Check that CONCURRENTLY is not specified if not populated. */
+ if (stmt->concurrent && !RelationIsPopulated(matviewRel))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
+
+ /* Check that conflicting options have not been specified. */
+ if (stmt->concurrent && stmt->skipData)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
+
+ /*
+ * Check that everything is correct for a refresh. Problems at this point
+ * are internal errors, so elog is sufficient.
+ */
+ if (matviewRel->rd_rel->relhasrules == false ||
+ matviewRel->rd_rules->numLocks < 1)
+ elog(ERROR,
+ "materialized view \"%s\" is missing rewrite information",
+ RelationGetRelationName(matviewRel));
+
+ if (matviewRel->rd_rules->numLocks > 1)
+ elog(ERROR,
+ "materialized view \"%s\" has too many rules",
+ RelationGetRelationName(matviewRel));
+
+ rule = matviewRel->rd_rules->rules[0];
+ if (rule->event != CMD_SELECT || !(rule->isInstead))
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
+ RelationGetRelationName(matviewRel));
+
+ actions = rule->actions;
+ if (list_length(actions) != 1)
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a single action",
+ RelationGetRelationName(matviewRel));
+
+ /*
+ * Check that there is a unique index with no WHERE clause on one or more
+ * columns of the materialized view if CONCURRENTLY is specified.
+ */
+ if (stmt->concurrent)
+ {
+ List *indexoidlist = RelationGetIndexList(matviewRel);
+ ListCell *indexoidscan;
+ bool hasUniqueIndex = false;
+
+ foreach(indexoidscan, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(indexoidscan);
+ Relation indexRel;
+
+ indexRel = index_open(indexoid, AccessShareLock);
+ hasUniqueIndex = is_usable_unique_index(indexRel);
+ index_close(indexRel, AccessShareLock);
+ if (hasUniqueIndex)
+ break;
+ }
+
+ list_free(indexoidlist);
+
+ if (!hasUniqueIndex)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot refresh materialized view \"%s\" concurrently",
+ quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
+ RelationGetRelationName(matviewRel))),
+ errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
+ }
+
+ /*
+ * The stored query was rewritten at the time of the MV definition, but
+ * has not been scribbled on by the planner.
+ */
+ dataQuery = linitial_node(Query, actions);
+
+ /*
+ * Check for active uses of the relation in the current transaction, such
+ * as open scans.
+ *
+ * NB: We count on this to protect us against problems with refreshing the
+ * data using TABLE_INSERT_FROZEN.
+ */
+ CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+
+ *rel = matviewRel;
+ *objectId = matviewOid;
+
+ return dataQuery;
+}
+
+/*
+ * get_new_heap_oid
+ *
+ * Create a new heap and return its oid to which the refresh materialized view
+ * data is inserted into.
+ */
+static Oid
+get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid matviewOid,
+ char *relpersistence)
+{
+ Oid OIDNewHeap;
+ Oid tableSpace;
+
+ /* Concurrent refresh builds new data in temp tablespace, and does diff. */
+ if (stmt->concurrent)
+ {
+ tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
+ *relpersistence = RELPERSISTENCE_TEMP;
+ }
+ else
+ {
+ tableSpace = matviewRel->rd_rel->reltablespace;
+ *relpersistence = matviewRel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Create the transient table that will receive the regenerated data. Lock
+ * it against access by any other process until commit (by which time it
+ * will be gone).
+ */
+ OIDNewHeap = make_new_heap(matviewOid, tableSpace, *relpersistence,
+ ExclusiveLock);
+ LockRelationOid(OIDNewHeap, AccessExclusiveLock);
+
+ return OIDNewHeap;
+}
+
+/*
+ * match_matview_with_new_data
+ *
+ * Arrange the materialized view newly generated data to match the existing
+ * data i.e merge in case of CONCURRENTLY otherwise perform heap swap and
+ * truncate the materialized view.
+ */
+static void
+match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, Oid OIDNewHeap, Oid relowner,
+ int save_sec_context, char relpersistence,
+ uint64 processed)
+{
+ /* Make the materialized view match the newly generated data. */
+ if (stmt->concurrent)
+ {
+ int old_depth = matview_maintenance_depth;
+
+ PG_TRY();
+ {
+ refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
+ save_sec_context);
+ }
+ PG_CATCH();
+ {
+ matview_maintenance_depth = old_depth;
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ Assert(matview_maintenance_depth == old_depth);
+ }
+ else
+ {
+ refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
+
+ /*
+ * Inform stats collector about our activity: basically, we truncated
+ * the materialized view and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the
+ * inserts and deletes it issues get counted by lower-level code.)
+ */
+ pgstat_count_truncate(matviewRel);
+ if (!stmt->skipData)
+ pgstat_count_heap_insert(matviewRel, processed);
+ }
+}
+
+/*
+ * rewrite_refresh_matview_query
+ *
+ * Rewrite the refresh materialized view data generating query.
+ *
+ * Work on the copied query to preserve the original query. Because the
+ * rewriter and planner tend to scribble on the input, we make a preliminary
+ * copy of the source querytree. This prevents problems in the case that
+ * REFRESH MATERIALIZED VIEW is in a portal or plpgsql function and is executed
+ * repeatedly. (See also the same hack in EXPLAIN and PREPARE.)
+ */
+static Query *
+rewrite_refresh_matview_query(Query *dataQuery)
+{
+ List *rewritten;
+ Query *copied_query;
+
+ /* Lock and rewrite, using a copy to preserve the original query. */
+ copied_query = copyObject(dataQuery);
+ AcquireRewriteLocks(copied_query, true, false);
+ rewritten = QueryRewrite(copied_query);
+
+ /* SELECT should never rewrite to more or less than one SELECT query */
+ if (list_length(rewritten) != 1)
+ elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
+ dataQuery = (Query *) linitial(rewritten);
+
+ return dataQuery;
+}
--
2.25.1
v5-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchapplication/x-patch; name=v5-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchDownload
From c627d596cc4ac06e406952ed97b3e417b857aedd Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Fri, 8 Jan 2021 14:15:46 +0530
Subject: [PATCH v5 2/2] EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
Currently, explain/explain analyze refresh materialized view(RMV)
is not allowed. We do plan the materialized view query before every
refresh. I propose to show the explain/explain analyze of the
select part of the materialized view. It will be useful for the
user to know what exactly is being planned and executed as part of
RMV. Please note that we already have explain/explain analyze
CTAS/Create Mat View(CMV), where we show the explain/explain analyze
of the select part. This proposal will do the same thing.
The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.
---
src/backend/commands/explain.c | 50 +++++++++++-----
src/backend/commands/matview.c | 36 +++++++++---
src/backend/commands/prepare.c | 3 +-
src/backend/tcop/utility.c | 3 +-
src/include/commands/explain.h | 30 +++++++++-
src/include/commands/matview.h | 8 ++-
src/test/regress/expected/matview.out | 83 +++++++++++++++++++++++++++
src/test/regress/sql/matview.sql | 27 +++++++++
8 files changed, 215 insertions(+), 25 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5d7eb3574c..7343a916b0 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -17,6 +17,7 @@
#include "catalog/pg_type.h"
#include "commands/createas.h"
#include "commands/defrem.h"
+#include "commands/matview.h"
#include "commands/prepare.h"
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
@@ -53,10 +54,6 @@ explain_get_index_name_hook_type explain_get_index_name_hook = NULL;
#define X_CLOSE_IMMEDIATE 2
#define X_NOWHITESPACE 4
-static void ExplainOneQuery(Query *query, int cursorOptions,
- IntoClause *into, ExplainState *es,
- const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv);
static void ExplainPrintJIT(ExplainState *es, int jit_flags,
JitInstrumentation *ji);
static void report_triggers(ResultRelInfo *rInfo, bool show_relname,
@@ -274,7 +271,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
{
ExplainOneQuery(lfirst_node(Query, l),
CURSOR_OPT_PARALLEL_OK, NULL, es,
- pstate->p_sourcetext, params, pstate->p_queryEnv);
+ pstate->p_sourcetext, params, pstate->p_queryEnv,
+ NULL);
/* Separate plans with an appropriate separator */
if (lnext(rewritten, l) != NULL)
@@ -357,11 +355,11 @@ ExplainResultDesc(ExplainStmt *stmt)
*
* "into" is NULL unless we are explaining the contents of a CreateTableAsStmt.
*/
-static void
+void
ExplainOneQuery(Query *query, int cursorOptions,
IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv)
+ QueryEnvironment *queryEnv, RefreshMatViewInfo *matviewInfo)
{
/* planner will not cope with utility statements */
if (query->commandType == CMD_UTILITY)
@@ -402,7 +400,8 @@ ExplainOneQuery(Query *query, int cursorOptions,
/* run it (if needed) and produce output */
ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ matviewInfo);
}
}
@@ -455,7 +454,7 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
CURSOR_OPT_PARALLEL_OK, ctas->into, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv, NULL);
}
else if (IsA(utilityStmt, DeclareCursorStmt))
{
@@ -474,7 +473,8 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
dcs->options, NULL, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv,
+ NULL);
}
else if (IsA(utilityStmt, ExecuteStmt))
ExplainExecuteQuery((ExecuteStmt *) utilityStmt, into, es,
@@ -486,6 +486,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
else
ExplainDummyGroup("Notify", NULL, es);
}
+ else if(IsA(utilityStmt, RefreshMatViewStmt))
+ {
+ RefreshMatViewExplainInfo explainInfo;
+
+ explainInfo.es = es;
+ explainInfo.queryEnv = queryEnv;
+
+ ExecRefreshMatView((RefreshMatViewStmt *) utilityStmt,
+ queryString, params, NULL, &explainInfo);
+ }
else
{
if (es->format == EXPLAIN_FORMAT_TEXT)
@@ -512,7 +522,7 @@ void
ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
- const BufferUsage *bufusage)
+ const BufferUsage *bufusage, RefreshMatViewInfo *matviewInfo)
{
DestReceiver *dest;
QueryDesc *queryDesc;
@@ -553,6 +563,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
if (into)
dest = CreateIntoRelDestReceiver(into);
+ else if (matviewInfo && OidIsValid(matviewInfo->OIDNewHeap))
+ dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);
else
dest = None_Receiver;
@@ -577,8 +589,12 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
{
ScanDirection dir;
- /* EXPLAIN ANALYZE CREATE TABLE AS WITH NO DATA is weird */
- if (into && into->skipData)
+ /*
+ * EXPLAIN ANALYZE CREATE TABLE AS or REFRESH MATERIALIZED VIEW
+ * WITH NO DATA is weird.
+ */
+ if ((into && into->skipData) ||
+ (matviewInfo && matviewInfo->skipData))
dir = NoMovementScanDirection;
else
dir = ForwardScanDirection;
@@ -586,6 +602,14 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
/* run the plan */
ExecutorRun(queryDesc, dir, 0L, true);
+ /*
+ * Collect the number of rows inserted in case of REFRESH MATERIALIZED
+ * VIEW which will be used while merging the newly generated data with
+ * the existing materialized view data in ExecRefreshMatView.
+ */
+ if (matviewInfo)
+ matviewInfo->processed = queryDesc->estate->es_processed;
+
/* run cleanup too */
ExecutorFinish(queryDesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 06bd5629a8..66b8c4b402 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -146,7 +146,8 @@ SetMatViewPopulatedState(Relation relation, bool newstate)
*/
ObjectAddress
ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc)
+ ParamListInfo params, QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo)
{
Oid matviewOid;
Relation matviewRel;
@@ -182,8 +183,11 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
- &relpersistence);
+ if (explainInfo && !explainInfo->es->analyze)
+ OIDNewHeap = InvalidOid;
+ else
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -192,17 +196,35 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_RESTRICTED_OPERATION);
/* Generate the data, if wanted. */
- if (!stmt->skipData)
+ if (!stmt->skipData && !explainInfo)
{
dataQuery = rewrite_refresh_matview_query(dataQuery);
processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
queryString);
}
+ else if (explainInfo)
+ {
+ RefreshMatViewInfo matViewInfo;
+
+ matViewInfo.OIDNewHeap = OIDNewHeap;
+ matViewInfo.skipData = stmt->skipData;
+ matViewInfo.processed = 0;
+
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
+
+ ExplainOneQuery(dataQuery,
+ CURSOR_OPT_PARALLEL_OK, NULL, explainInfo->es,
+ queryString, params, explainInfo->queryEnv,
+ &matViewInfo);
+
+ processed = matViewInfo.processed;
+ }
- match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
- relowner, save_sec_context, relpersistence,
- processed);
+ if (OidIsValid(OIDNewHeap))
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 653ef8e41a..696d3343d4 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -672,7 +672,8 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
if (pstmt->commandType != CMD_UTILITY)
ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ NULL);
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 53a511f1da..ae7cef9c24 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1659,7 +1659,8 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
address = ExecRefreshMatView((RefreshMatViewStmt *) parsetree,
- queryString, params, qc);
+ queryString, params, qc,
+ NULL);
}
PG_FINALLY();
{
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7f27d92df1 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -61,6 +61,27 @@ typedef struct ExplainState
ExplainWorkersState *workers_state; /* needed if parallel plan */
} ExplainState;
+/*
+ * Refresh Materialized View information passed across functions for EXPLAIN
+ * execution.
+ */
+typedef struct RefreshMatViewInfo
+{
+ /* Oid of the new heap created. */
+ Oid OIDNewHeap;
+ /* Is WITH NO DATA clause specified? */
+ bool skipData;
+ /* Number of rows inserted. */
+ uint64 processed;
+} RefreshMatViewInfo;
+
+/* EXPLAIN information shared to ExecRefreshMatView(). */
+typedef struct RefreshMatViewExplainInfo
+{
+ ExplainState *es;
+ QueryEnvironment *queryEnv;
+} RefreshMatViewExplainInfo;
+
/* Hook for plugins to get control in ExplainOneQuery() */
typedef void (*ExplainOneQuery_hook_type) (Query *query,
int cursorOptions,
@@ -91,7 +112,14 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
- const BufferUsage *bufusage);
+ const BufferUsage *bufusage,
+ RefreshMatViewInfo *matviewInfo);
+
+extern void ExplainOneQuery(Query *query, int cursorOptions,
+ IntoClause *into, ExplainState *es,
+ const char *queryString, ParamListInfo params,
+ QueryEnvironment *queryEnv,
+ RefreshMatViewInfo *matviewInfo);
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
diff --git a/src/include/commands/matview.h b/src/include/commands/matview.h
index 214b1c1df6..1d60180ebc 100644
--- a/src/include/commands/matview.h
+++ b/src/include/commands/matview.h
@@ -15,6 +15,7 @@
#define MATVIEW_H
#include "catalog/objectaddress.h"
+#include "commands/explain.h"
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "tcop/dest.h"
@@ -23,8 +24,11 @@
extern void SetMatViewPopulatedState(Relation relation, bool newstate);
-extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc);
+extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt,
+ const char *queryString,
+ ParamListInfo params,
+ QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo);
extern DestReceiver *CreateTransientRelDestReceiver(Oid oid);
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..305e511fea 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -668,3 +668,86 @@ NOTICE: relation "matview_ine_tab" already exists, skipping
(0 rows)
DROP MATERIALIZED VIEW matview_ine_tab;
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+-----------------------------------------
+ Seq Scan on mv_exp_tbl (never executed)
+ Filter: (a > 5)
+(2 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+ERROR: materialized view "mv_exp" has not been populated
+HINT: Use the REFRESH MATERIALIZED VIEW command.
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
diff --git a/src/test/regress/sql/matview.sql b/src/test/regress/sql/matview.sql
index 4a4bd0d6b6..67a45bbde7 100644
--- a/src/test/regress/sql/matview.sql
+++ b/src/test/regress/sql/matview.sql
@@ -287,3 +287,30 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
CREATE MATERIALIZED VIEW IF NOT EXISTS matview_ine_tab AS
SELECT 1 / 0 WITH NO DATA; -- ok
DROP MATERIALIZED VIEW matview_ine_tab;
+
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
--
2.25.1
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Fri, 05 Mar 2021 at 19:48, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
Attaching v5 patch set for further review.
The v5 patch looks good to me, if there is no objection, I'll change the
cf status to "Ready for Committer" in few days.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Sun, Mar 7, 2021 at 11:49 AM Japin Li <japinli@hotmail.com> wrote:
On Fri, 05 Mar 2021 at 19:48, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
Attaching v5 patch set for further review.
The v5 patch looks good to me, if there is no objection, I'll change the
cf status to "Ready for Committer" in few days.
Thanks for the review.
As I mentioned upthread, I have 2 open points:
1) In the patch I have added a new mat view info parameter to
ExplainOneQuery(), do we also need to add it to
ExplainOneQuery_hook_type? IMO, we should not (for now), because this
would create a backward compatibility issue.
2) Do we document (under respective command pages or somewhere else)
that we allow explain/explain analyze for a command?
Thoughts?
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Sun, 07 Mar 2021 at 14:25, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
On Sun, Mar 7, 2021 at 11:49 AM Japin Li <japinli@hotmail.com> wrote:
On Fri, 05 Mar 2021 at 19:48, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
Attaching v5 patch set for further review.
The v5 patch looks good to me, if there is no objection, I'll change the
cf status to "Ready for Committer" in few days.Thanks for the review.
As I mentioned upthread, I have 2 open points:
1) In the patch I have added a new mat view info parameter to
ExplainOneQuery(), do we also need to add it to
ExplainOneQuery_hook_type? IMO, we should not (for now), because this
would create a backward compatibility issue.
Sorry, I do not know how PostgreSQL handle the backward compatibility issue.
Is there a guideline?
2) Do we document (under respective command pages or somewhere else)
that we allow explain/explain analyze for a command?
IMO, we can add a new page to list the commands that can be explain/explain analyze,
since it's clear for users.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Sun, Mar 7, 2021 at 12:13 PM Japin Li <japinli@hotmail.com> wrote:
On Sun, 07 Mar 2021 at 14:25, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
On Sun, Mar 7, 2021 at 11:49 AM Japin Li <japinli@hotmail.com> wrote:
On Fri, 05 Mar 2021 at 19:48, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
Attaching v5 patch set for further review.
The v5 patch looks good to me, if there is no objection, I'll change the
cf status to "Ready for Committer" in few days.Thanks for the review.
As I mentioned upthread, I have 2 open points:
1) In the patch I have added a new mat view info parameter to
ExplainOneQuery(), do we also need to add it to
ExplainOneQuery_hook_type? IMO, we should not (for now), because this
would create a backward compatibility issue.Sorry, I do not know how PostgreSQL handle the backward compatibility issue.
Is there a guideline?
I'm not aware of any guidelines as such, but we usually avoid any
changes to existing API, adding/making changes to system catalogs and
so on.
2) Do we document (under respective command pages or somewhere else)
that we allow explain/explain analyze for a command?IMO, we can add a new page to list the commands that can be explain/explain analyze,
since it's clear for users.
We are listing all the supported commands in explain.sgml, so added
the CREATE MATERIALIZED VIEW(it's missing even though it's supported
prior to this patch) and REFRESH MATERIALIZED VIEW there.
Attaching v6 patch set. Please have a look.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v6-0001-Rearrange-Refresh-Mat-View-Code.patchapplication/x-patch; name=v6-0001-Rearrange-Refresh-Mat-View-Code.patchDownload
From 26970ffd33e67324609a03f0f61eeb6406216a7f Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Fri, 5 Mar 2021 15:47:12 +0530
Subject: [PATCH v5 1/2] Rearrange Refresh Mat View Code
Currently, the function ExecRefreshMatView in matview.c is having
many lines of code which is not at all good from readability and
maintainability perspectives. This patch adds few functions and
moves the code from ExecRefreshMatView to them making the code
look better.
---
src/backend/commands/matview.c | 449 ++++++++++++++++++++-------------
1 file changed, 269 insertions(+), 180 deletions(-)
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..18e18fa627 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -64,7 +64,7 @@ static void transientrel_startup(DestReceiver *self, int operation, TupleDesc ty
static bool transientrel_receive(TupleTableSlot *slot, DestReceiver *self);
static void transientrel_shutdown(DestReceiver *self);
static void transientrel_destroy(DestReceiver *self);
-static uint64 refresh_matview_datafill(DestReceiver *dest, Query *query,
+static uint64 refresh_matview_datafill(Oid OIDNewHeap, Query *query,
const char *queryString);
static char *make_temptable_name_n(char *tempname, int n);
static void refresh_by_match_merge(Oid matviewOid, Oid tempOid, Oid relowner,
@@ -73,6 +73,16 @@ static void refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersist
static bool is_usable_unique_index(Relation indexRel);
static void OpenMatViewIncrementalMaintenance(void);
static void CloseMatViewIncrementalMaintenance(void);
+static Query *get_matview_query(RefreshMatViewStmt *stmt, Relation *rel,
+ Oid *objectId);
+static Query *rewrite_refresh_matview_query(Query *dataQuery);
+static Oid get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, char *relpersistence);
+static void match_matview_with_new_data(RefreshMatViewStmt *stmt,
+ Relation matviewRel, Oid matviewOid,
+ Oid OIDNewHeap, Oid relowner,
+ int save_sec_context,
+ char relpersistence, uint64 processed);
/*
* SetMatViewPopulatedState
@@ -140,127 +150,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
{
Oid matviewOid;
Relation matviewRel;
- RewriteRule *rule;
- List *actions;
Query *dataQuery;
- Oid tableSpace;
- Oid relowner;
Oid OIDNewHeap;
- DestReceiver *dest;
uint64 processed = 0;
- bool concurrent;
- LOCKMODE lockmode;
+ Oid relowner;
char relpersistence;
Oid save_userid;
int save_sec_context;
int save_nestlevel;
ObjectAddress address;
- /* Determine strength of lock needed. */
- concurrent = stmt->concurrent;
- lockmode = concurrent ? ExclusiveLock : AccessExclusiveLock;
-
- /*
- * Get a lock until end of transaction.
- */
- matviewOid = RangeVarGetRelidExtended(stmt->relation,
- lockmode, 0,
- RangeVarCallbackOwnsTable, NULL);
- matviewRel = table_open(matviewOid, NoLock);
-
- /* Make sure it is a materialized view. */
- if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("\"%s\" is not a materialized view",
- RelationGetRelationName(matviewRel))));
-
- /* Check that CONCURRENTLY is not specified if not populated. */
- if (concurrent && !RelationIsPopulated(matviewRel))
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
-
- /* Check that conflicting options have not been specified. */
- if (concurrent && stmt->skipData)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
-
- /*
- * Check that everything is correct for a refresh. Problems at this point
- * are internal errors, so elog is sufficient.
- */
- if (matviewRel->rd_rel->relhasrules == false ||
- matviewRel->rd_rules->numLocks < 1)
- elog(ERROR,
- "materialized view \"%s\" is missing rewrite information",
- RelationGetRelationName(matviewRel));
-
- if (matviewRel->rd_rules->numLocks > 1)
- elog(ERROR,
- "materialized view \"%s\" has too many rules",
- RelationGetRelationName(matviewRel));
-
- rule = matviewRel->rd_rules->rules[0];
- if (rule->event != CMD_SELECT || !(rule->isInstead))
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
- RelationGetRelationName(matviewRel));
-
- actions = rule->actions;
- if (list_length(actions) != 1)
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a single action",
- RelationGetRelationName(matviewRel));
-
- /*
- * Check that there is a unique index with no WHERE clause on one or more
- * columns of the materialized view if CONCURRENTLY is specified.
- */
- if (concurrent)
- {
- List *indexoidlist = RelationGetIndexList(matviewRel);
- ListCell *indexoidscan;
- bool hasUniqueIndex = false;
-
- foreach(indexoidscan, indexoidlist)
- {
- Oid indexoid = lfirst_oid(indexoidscan);
- Relation indexRel;
-
- indexRel = index_open(indexoid, AccessShareLock);
- hasUniqueIndex = is_usable_unique_index(indexRel);
- index_close(indexRel, AccessShareLock);
- if (hasUniqueIndex)
- break;
- }
-
- list_free(indexoidlist);
-
- if (!hasUniqueIndex)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot refresh materialized view \"%s\" concurrently",
- quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
- RelationGetRelationName(matviewRel))),
- errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
- }
-
- /*
- * The stored query was rewritten at the time of the MV definition, but
- * has not been scribbled on by the planner.
- */
- dataQuery = linitial_node(Query, actions);
-
- /*
- * Check for active uses of the relation in the current transaction, such
- * as open scans.
- *
- * NB: We count on this to protect us against problems with refreshing the
- * data using TABLE_INSERT_FROZEN.
- */
- CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+ /* Get the data generating query. */
+ dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);
/*
* Tentatively mark the matview as populated or not (this will roll back
@@ -281,27 +182,8 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- /* Concurrent refresh builds new data in temp tablespace, and does diff. */
- if (concurrent)
- {
- tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
- relpersistence = RELPERSISTENCE_TEMP;
- }
- else
- {
- tableSpace = matviewRel->rd_rel->reltablespace;
- relpersistence = matviewRel->rd_rel->relpersistence;
- }
-
- /*
- * Create the transient table that will receive the regenerated data. Lock
- * it against access by any other process until commit (by which time it
- * will be gone).
- */
- OIDNewHeap = make_new_heap(matviewOid, tableSpace, relpersistence,
- ExclusiveLock);
- LockRelationOid(OIDNewHeap, AccessExclusiveLock);
- dest = CreateTransientRelDestReceiver(OIDNewHeap);
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -311,40 +193,16 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/* Generate the data, if wanted. */
if (!stmt->skipData)
- processed = refresh_matview_datafill(dest, dataQuery, queryString);
-
- /* Make the matview match the newly generated data. */
- if (concurrent)
{
- int old_depth = matview_maintenance_depth;
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
- PG_TRY();
- {
- refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
- save_sec_context);
- }
- PG_CATCH();
- {
- matview_maintenance_depth = old_depth;
- PG_RE_THROW();
- }
- PG_END_TRY();
- Assert(matview_maintenance_depth == old_depth);
+ processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
+ queryString);
}
- else
- {
- refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
- /*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
- */
- pgstat_count_truncate(matviewRel);
- if (!stmt->skipData)
- pgstat_count_heap_insert(matviewRel, processed);
- }
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
@@ -373,30 +231,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/*
* refresh_matview_datafill
*
- * Execute the given query, sending result rows to "dest" (which will
- * insert them into the target matview).
+ * Create dest receiver and execute the given query, sending result rows to the
+ * dest receiver which will insert them into the target materialized view.
*
* Returns number of rows inserted.
*/
static uint64
-refresh_matview_datafill(DestReceiver *dest, Query *query,
- const char *queryString)
+refresh_matview_datafill(Oid OIDNewHeap, Query *query, const char *queryString)
{
- List *rewritten;
PlannedStmt *plan;
QueryDesc *queryDesc;
- Query *copied_query;
uint64 processed;
-
- /* Lock and rewrite, using a copy to preserve the original query. */
- copied_query = copyObject(query);
- AcquireRewriteLocks(copied_query, true, false);
- rewritten = QueryRewrite(copied_query);
-
- /* SELECT should never rewrite to more or less than one SELECT query */
- if (list_length(rewritten) != 1)
- elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
- query = (Query *) linitial(rewritten);
+ DestReceiver *dest;
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
@@ -413,6 +259,8 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
PushCopiedSnapshot(GetActiveSnapshot());
UpdateActiveSnapshotCommandId();
+ dest = CreateTransientRelDestReceiver(OIDNewHeap);
+
/* Create a QueryDesc, redirecting output to our tuple receiver */
queryDesc = CreateQueryDesc(plan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
@@ -933,3 +781,244 @@ CloseMatViewIncrementalMaintenance(void)
matview_maintenance_depth--;
Assert(matview_maintenance_depth >= 0);
}
+
+/*
+ * get_matview_query
+ *
+ * Open the refresh materialized view relation, perform sanity checks and also
+ * get the associated data generating query from it.
+ *
+ * Note that the refresh materialized view relation is opened here, it has to
+ * be closed in the caller.
+ */
+static Query *
+get_matview_query(RefreshMatViewStmt *stmt, Relation *rel, Oid *objectId)
+{
+ Oid matviewOid;
+ Relation matviewRel;
+ RewriteRule *rule;
+ List *actions;
+ Query *dataQuery;
+ LOCKMODE lockmode;
+
+ /* Determine strength of lock needed. */
+ lockmode = stmt->concurrent ? ExclusiveLock : AccessExclusiveLock;
+
+ /* Get a lock until end of transaction. */
+ matviewOid = RangeVarGetRelidExtended(stmt->relation, lockmode, 0,
+ RangeVarCallbackOwnsTable, NULL);
+ matviewRel = table_open(matviewOid, NoLock);
+
+ /* Make sure it is a materialized view. */
+ if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("\"%s\" is not a materialized view",
+ RelationGetRelationName(matviewRel))));
+
+ /* Check that CONCURRENTLY is not specified if not populated. */
+ if (stmt->concurrent && !RelationIsPopulated(matviewRel))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
+
+ /* Check that conflicting options have not been specified. */
+ if (stmt->concurrent && stmt->skipData)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
+
+ /*
+ * Check that everything is correct for a refresh. Problems at this point
+ * are internal errors, so elog is sufficient.
+ */
+ if (matviewRel->rd_rel->relhasrules == false ||
+ matviewRel->rd_rules->numLocks < 1)
+ elog(ERROR,
+ "materialized view \"%s\" is missing rewrite information",
+ RelationGetRelationName(matviewRel));
+
+ if (matviewRel->rd_rules->numLocks > 1)
+ elog(ERROR,
+ "materialized view \"%s\" has too many rules",
+ RelationGetRelationName(matviewRel));
+
+ rule = matviewRel->rd_rules->rules[0];
+ if (rule->event != CMD_SELECT || !(rule->isInstead))
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
+ RelationGetRelationName(matviewRel));
+
+ actions = rule->actions;
+ if (list_length(actions) != 1)
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a single action",
+ RelationGetRelationName(matviewRel));
+
+ /*
+ * Check that there is a unique index with no WHERE clause on one or more
+ * columns of the materialized view if CONCURRENTLY is specified.
+ */
+ if (stmt->concurrent)
+ {
+ List *indexoidlist = RelationGetIndexList(matviewRel);
+ ListCell *indexoidscan;
+ bool hasUniqueIndex = false;
+
+ foreach(indexoidscan, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(indexoidscan);
+ Relation indexRel;
+
+ indexRel = index_open(indexoid, AccessShareLock);
+ hasUniqueIndex = is_usable_unique_index(indexRel);
+ index_close(indexRel, AccessShareLock);
+ if (hasUniqueIndex)
+ break;
+ }
+
+ list_free(indexoidlist);
+
+ if (!hasUniqueIndex)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot refresh materialized view \"%s\" concurrently",
+ quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
+ RelationGetRelationName(matviewRel))),
+ errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
+ }
+
+ /*
+ * The stored query was rewritten at the time of the MV definition, but
+ * has not been scribbled on by the planner.
+ */
+ dataQuery = linitial_node(Query, actions);
+
+ /*
+ * Check for active uses of the relation in the current transaction, such
+ * as open scans.
+ *
+ * NB: We count on this to protect us against problems with refreshing the
+ * data using TABLE_INSERT_FROZEN.
+ */
+ CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+
+ *rel = matviewRel;
+ *objectId = matviewOid;
+
+ return dataQuery;
+}
+
+/*
+ * get_new_heap_oid
+ *
+ * Create a new heap and return its oid to which the refresh materialized view
+ * data is inserted into.
+ */
+static Oid
+get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid matviewOid,
+ char *relpersistence)
+{
+ Oid OIDNewHeap;
+ Oid tableSpace;
+
+ /* Concurrent refresh builds new data in temp tablespace, and does diff. */
+ if (stmt->concurrent)
+ {
+ tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
+ *relpersistence = RELPERSISTENCE_TEMP;
+ }
+ else
+ {
+ tableSpace = matviewRel->rd_rel->reltablespace;
+ *relpersistence = matviewRel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Create the transient table that will receive the regenerated data. Lock
+ * it against access by any other process until commit (by which time it
+ * will be gone).
+ */
+ OIDNewHeap = make_new_heap(matviewOid, tableSpace, *relpersistence,
+ ExclusiveLock);
+ LockRelationOid(OIDNewHeap, AccessExclusiveLock);
+
+ return OIDNewHeap;
+}
+
+/*
+ * match_matview_with_new_data
+ *
+ * Arrange the materialized view newly generated data to match the existing
+ * data i.e merge in case of CONCURRENTLY otherwise perform heap swap and
+ * truncate the materialized view.
+ */
+static void
+match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, Oid OIDNewHeap, Oid relowner,
+ int save_sec_context, char relpersistence,
+ uint64 processed)
+{
+ /* Make the materialized view match the newly generated data. */
+ if (stmt->concurrent)
+ {
+ int old_depth = matview_maintenance_depth;
+
+ PG_TRY();
+ {
+ refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
+ save_sec_context);
+ }
+ PG_CATCH();
+ {
+ matview_maintenance_depth = old_depth;
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ Assert(matview_maintenance_depth == old_depth);
+ }
+ else
+ {
+ refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
+
+ /*
+ * Inform stats collector about our activity: basically, we truncated
+ * the materialized view and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the
+ * inserts and deletes it issues get counted by lower-level code.)
+ */
+ pgstat_count_truncate(matviewRel);
+ if (!stmt->skipData)
+ pgstat_count_heap_insert(matviewRel, processed);
+ }
+}
+
+/*
+ * rewrite_refresh_matview_query
+ *
+ * Rewrite the refresh materialized view data generating query.
+ *
+ * Work on the copied query to preserve the original query. Because the
+ * rewriter and planner tend to scribble on the input, we make a preliminary
+ * copy of the source querytree. This prevents problems in the case that
+ * REFRESH MATERIALIZED VIEW is in a portal or plpgsql function and is executed
+ * repeatedly. (See also the same hack in EXPLAIN and PREPARE.)
+ */
+static Query *
+rewrite_refresh_matview_query(Query *dataQuery)
+{
+ List *rewritten;
+ Query *copied_query;
+
+ /* Lock and rewrite, using a copy to preserve the original query. */
+ copied_query = copyObject(dataQuery);
+ AcquireRewriteLocks(copied_query, true, false);
+ rewritten = QueryRewrite(copied_query);
+
+ /* SELECT should never rewrite to more or less than one SELECT query */
+ if (list_length(rewritten) != 1)
+ elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
+ dataQuery = (Query *) linitial(rewritten);
+
+ return dataQuery;
+}
--
2.25.1
v6-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchapplication/x-patch; name=v6-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchDownload
From 08906356cbbd1fe1231d633847552290e9155ec5 Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Sun, 7 Mar 2021 14:57:24 +0530
Subject: [PATCH v6] EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
Currently, explain/explain analyze refresh materialized view(RMV)
is not allowed. We do plan the materialized view query before every
refresh. I propose to show the explain/explain analyze of the
select part of the materialized view. It will be useful for the
user to know what exactly is being planned and executed as part of
RMV. Please note that we already have explain/explain analyze
CTAS/Create Mat View(CMV), where we show the explain/explain analyze
of the select part. This proposal will do the same thing.
The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.
---
doc/src/sgml/ref/explain.sgml | 2 +
src/backend/commands/explain.c | 50 +++++++++++-----
src/backend/commands/matview.c | 36 +++++++++---
src/backend/commands/prepare.c | 3 +-
src/backend/tcop/utility.c | 3 +-
src/include/commands/explain.h | 30 +++++++++-
src/include/commands/matview.h | 8 ++-
src/test/regress/expected/matview.out | 83 +++++++++++++++++++++++++++
src/test/regress/sql/matview.sql | 27 +++++++++
9 files changed, 217 insertions(+), 25 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index c4512332a0..a301b6b53f 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -95,6 +95,8 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
<command>EXPLAIN ANALYZE</command> on an
<command>INSERT</command>, <command>UPDATE</command>,
<command>DELETE</command>, <command>CREATE TABLE AS</command>,
+ <command>CREATE MATERIALIZED VIEW</command>,
+ <command>REFRESH MATERIALIZED VIEW</command>,
or <command>EXECUTE</command> statement
without letting the command affect your data, use this approach:
<programlisting>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index afc45429ba..ac15931a41 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -17,6 +17,7 @@
#include "catalog/pg_type.h"
#include "commands/createas.h"
#include "commands/defrem.h"
+#include "commands/matview.h"
#include "commands/prepare.h"
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
@@ -53,10 +54,6 @@ explain_get_index_name_hook_type explain_get_index_name_hook = NULL;
#define X_CLOSE_IMMEDIATE 2
#define X_NOWHITESPACE 4
-static void ExplainOneQuery(Query *query, int cursorOptions,
- IntoClause *into, ExplainState *es,
- const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv);
static void ExplainPrintJIT(ExplainState *es, int jit_flags,
JitInstrumentation *ji);
static void report_triggers(ResultRelInfo *rInfo, bool show_relname,
@@ -274,7 +271,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
{
ExplainOneQuery(lfirst_node(Query, l),
CURSOR_OPT_PARALLEL_OK, NULL, es,
- pstate->p_sourcetext, params, pstate->p_queryEnv);
+ pstate->p_sourcetext, params, pstate->p_queryEnv,
+ NULL);
/* Separate plans with an appropriate separator */
if (lnext(rewritten, l) != NULL)
@@ -357,11 +355,11 @@ ExplainResultDesc(ExplainStmt *stmt)
*
* "into" is NULL unless we are explaining the contents of a CreateTableAsStmt.
*/
-static void
+void
ExplainOneQuery(Query *query, int cursorOptions,
IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv)
+ QueryEnvironment *queryEnv, RefreshMatViewInfo *matviewInfo)
{
/* planner will not cope with utility statements */
if (query->commandType == CMD_UTILITY)
@@ -402,7 +400,8 @@ ExplainOneQuery(Query *query, int cursorOptions,
/* run it (if needed) and produce output */
ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ matviewInfo);
}
}
@@ -455,7 +454,7 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
CURSOR_OPT_PARALLEL_OK, ctas->into, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv, NULL);
}
else if (IsA(utilityStmt, DeclareCursorStmt))
{
@@ -474,7 +473,8 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
dcs->options, NULL, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv,
+ NULL);
}
else if (IsA(utilityStmt, ExecuteStmt))
ExplainExecuteQuery((ExecuteStmt *) utilityStmt, into, es,
@@ -486,6 +486,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
else
ExplainDummyGroup("Notify", NULL, es);
}
+ else if(IsA(utilityStmt, RefreshMatViewStmt))
+ {
+ RefreshMatViewExplainInfo explainInfo;
+
+ explainInfo.es = es;
+ explainInfo.queryEnv = queryEnv;
+
+ ExecRefreshMatView((RefreshMatViewStmt *) utilityStmt,
+ queryString, params, NULL, &explainInfo);
+ }
else
{
if (es->format == EXPLAIN_FORMAT_TEXT)
@@ -512,7 +522,7 @@ void
ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
- const BufferUsage *bufusage)
+ const BufferUsage *bufusage, RefreshMatViewInfo *matviewInfo)
{
DestReceiver *dest;
QueryDesc *queryDesc;
@@ -553,6 +563,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
if (into)
dest = CreateIntoRelDestReceiver(into);
+ else if (matviewInfo && OidIsValid(matviewInfo->OIDNewHeap))
+ dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);
else
dest = None_Receiver;
@@ -577,8 +589,12 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
{
ScanDirection dir;
- /* EXPLAIN ANALYZE CREATE TABLE AS WITH NO DATA is weird */
- if (into && into->skipData)
+ /*
+ * EXPLAIN ANALYZE CREATE TABLE AS or REFRESH MATERIALIZED VIEW
+ * WITH NO DATA is weird.
+ */
+ if ((into && into->skipData) ||
+ (matviewInfo && matviewInfo->skipData))
dir = NoMovementScanDirection;
else
dir = ForwardScanDirection;
@@ -586,6 +602,14 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
/* run the plan */
ExecutorRun(queryDesc, dir, 0L, true);
+ /*
+ * Collect the number of rows inserted in case of REFRESH MATERIALIZED
+ * VIEW which will be used while merging the newly generated data with
+ * the existing materialized view data in ExecRefreshMatView.
+ */
+ if (matviewInfo)
+ matviewInfo->processed = queryDesc->estate->es_processed;
+
/* run cleanup too */
ExecutorFinish(queryDesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 18e18fa627..5c44d6b451 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -146,7 +146,8 @@ SetMatViewPopulatedState(Relation relation, bool newstate)
*/
ObjectAddress
ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc)
+ ParamListInfo params, QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo)
{
Oid matviewOid;
Relation matviewRel;
@@ -182,8 +183,11 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
- &relpersistence);
+ if (explainInfo && !explainInfo->es->analyze)
+ OIDNewHeap = InvalidOid;
+ else
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -192,17 +196,35 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_RESTRICTED_OPERATION);
/* Generate the data, if wanted. */
- if (!stmt->skipData)
+ if (!stmt->skipData && !explainInfo)
{
dataQuery = rewrite_refresh_matview_query(dataQuery);
processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
queryString);
}
+ else if (explainInfo)
+ {
+ RefreshMatViewInfo matViewInfo;
+
+ matViewInfo.OIDNewHeap = OIDNewHeap;
+ matViewInfo.skipData = stmt->skipData;
+ matViewInfo.processed = 0;
+
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
+
+ ExplainOneQuery(dataQuery,
+ CURSOR_OPT_PARALLEL_OK, NULL, explainInfo->es,
+ queryString, params, explainInfo->queryEnv,
+ &matViewInfo);
+
+ processed = matViewInfo.processed;
+ }
- match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
- relowner, save_sec_context, relpersistence,
- processed);
+ if (OidIsValid(OIDNewHeap))
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index f767751c71..56c7432879 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -673,7 +673,8 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
if (pstmt->commandType != CMD_UTILITY)
ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ NULL);
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 05bb698cf4..20087addb4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1621,7 +1621,8 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
address = ExecRefreshMatView((RefreshMatViewStmt *) parsetree,
- queryString, params, qc);
+ queryString, params, qc,
+ NULL);
}
PG_FINALLY();
{
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7f27d92df1 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -61,6 +61,27 @@ typedef struct ExplainState
ExplainWorkersState *workers_state; /* needed if parallel plan */
} ExplainState;
+/*
+ * Refresh Materialized View information passed across functions for EXPLAIN
+ * execution.
+ */
+typedef struct RefreshMatViewInfo
+{
+ /* Oid of the new heap created. */
+ Oid OIDNewHeap;
+ /* Is WITH NO DATA clause specified? */
+ bool skipData;
+ /* Number of rows inserted. */
+ uint64 processed;
+} RefreshMatViewInfo;
+
+/* EXPLAIN information shared to ExecRefreshMatView(). */
+typedef struct RefreshMatViewExplainInfo
+{
+ ExplainState *es;
+ QueryEnvironment *queryEnv;
+} RefreshMatViewExplainInfo;
+
/* Hook for plugins to get control in ExplainOneQuery() */
typedef void (*ExplainOneQuery_hook_type) (Query *query,
int cursorOptions,
@@ -91,7 +112,14 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
- const BufferUsage *bufusage);
+ const BufferUsage *bufusage,
+ RefreshMatViewInfo *matviewInfo);
+
+extern void ExplainOneQuery(Query *query, int cursorOptions,
+ IntoClause *into, ExplainState *es,
+ const char *queryString, ParamListInfo params,
+ QueryEnvironment *queryEnv,
+ RefreshMatViewInfo *matviewInfo);
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
diff --git a/src/include/commands/matview.h b/src/include/commands/matview.h
index 214b1c1df6..1d60180ebc 100644
--- a/src/include/commands/matview.h
+++ b/src/include/commands/matview.h
@@ -15,6 +15,7 @@
#define MATVIEW_H
#include "catalog/objectaddress.h"
+#include "commands/explain.h"
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "tcop/dest.h"
@@ -23,8 +24,11 @@
extern void SetMatViewPopulatedState(Relation relation, bool newstate);
-extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc);
+extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt,
+ const char *queryString,
+ ParamListInfo params,
+ QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo);
extern DestReceiver *CreateTransientRelDestReceiver(Oid oid);
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..305e511fea 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -668,3 +668,86 @@ NOTICE: relation "matview_ine_tab" already exists, skipping
(0 rows)
DROP MATERIALIZED VIEW matview_ine_tab;
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+-----------------------------------------
+ Seq Scan on mv_exp_tbl (never executed)
+ Filter: (a > 5)
+(2 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+ERROR: materialized view "mv_exp" has not been populated
+HINT: Use the REFRESH MATERIALIZED VIEW command.
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
diff --git a/src/test/regress/sql/matview.sql b/src/test/regress/sql/matview.sql
index 4a4bd0d6b6..67a45bbde7 100644
--- a/src/test/regress/sql/matview.sql
+++ b/src/test/regress/sql/matview.sql
@@ -287,3 +287,30 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
CREATE MATERIALIZED VIEW IF NOT EXISTS matview_ine_tab AS
SELECT 1 / 0 WITH NO DATA; -- ok
DROP MATERIALIZED VIEW matview_ine_tab;
+
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
--
2.25.1
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Sun, 07 Mar 2021 at 17:33, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
On Sun, Mar 7, 2021 at 12:13 PM Japin Li <japinli@hotmail.com> wrote:
On Sun, 07 Mar 2021 at 14:25, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
On Sun, Mar 7, 2021 at 11:49 AM Japin Li <japinli@hotmail.com> wrote:
On Fri, 05 Mar 2021 at 19:48, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
Attaching v5 patch set for further review.
The v5 patch looks good to me, if there is no objection, I'll change the
cf status to "Ready for Committer" in few days.Thanks for the review.
As I mentioned upthread, I have 2 open points:
1) In the patch I have added a new mat view info parameter to
ExplainOneQuery(), do we also need to add it to
ExplainOneQuery_hook_type? IMO, we should not (for now), because this
would create a backward compatibility issue.Sorry, I do not know how PostgreSQL handle the backward compatibility issue.
Is there a guideline?I'm not aware of any guidelines as such, but we usually avoid any
changes to existing API, adding/making changes to system catalogs and
so on.
Thanks for explaining, I'd be inclined keep it backward compatibility.
2) Do we document (under respective command pages or somewhere else)
that we allow explain/explain analyze for a command?IMO, we can add a new page to list the commands that can be explain/explain analyze,
since it's clear for users.We are listing all the supported commands in explain.sgml, so added
the CREATE MATERIALIZED VIEW(it's missing even though it's supported
prior to this patch) and REFRESH MATERIALIZED VIEW there.Attaching v6 patch set. Please have a look.
LGTM.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
Hi,
+ * EXPLAIN ANALYZE CREATE TABLE AS or REFRESH MATERIALIZED VIEW
+ * WITH NO DATA is weird.Maybe it is clearer to spell out WITH NO DATA for both statements, instead
of sharing it.
- if (!stmt->skipData)
+ if (!stmt->skipData && !explainInfo)
...
+ else if (explainInfo)It would be cleaner to put the 'if (explainInfo)' as the first check. That
way, the check for skipData can be simplified.
Cheers
On Sun, Mar 7, 2021 at 1:34 AM Bharath Rupireddy <
bharath.rupireddyforpostgres@gmail.com> wrote:
Show quoted text
On Sun, Mar 7, 2021 at 12:13 PM Japin Li <japinli@hotmail.com> wrote:
On Sun, 07 Mar 2021 at 14:25, Bharath Rupireddy <
bharath.rupireddyforpostgres@gmail.com> wrote:
On Sun, Mar 7, 2021 at 11:49 AM Japin Li <japinli@hotmail.com> wrote:
On Fri, 05 Mar 2021 at 19:48, Bharath Rupireddy <
bharath.rupireddyforpostgres@gmail.com> wrote:
Attaching v5 patch set for further review.
The v5 patch looks good to me, if there is no objection, I'll change
the
cf status to "Ready for Committer" in few days.
Thanks for the review.
As I mentioned upthread, I have 2 open points:
1) In the patch I have added a new mat view info parameter to
ExplainOneQuery(), do we also need to add it to
ExplainOneQuery_hook_type? IMO, we should not (for now), because this
would create a backward compatibility issue.Sorry, I do not know how PostgreSQL handle the backward compatibility
issue.
Is there a guideline?
I'm not aware of any guidelines as such, but we usually avoid any
changes to existing API, adding/making changes to system catalogs and
so on.2) Do we document (under respective command pages or somewhere else)
that we allow explain/explain analyze for a command?IMO, we can add a new page to list the commands that can be
explain/explain analyze,
since it's clear for users.
We are listing all the supported commands in explain.sgml, so added
the CREATE MATERIALIZED VIEW(it's missing even though it's supported
prior to this patch) and REFRESH MATERIALIZED VIEW there.Attaching v6 patch set. Please have a look.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Sun, Mar 7, 2021 at 10:13 PM Zhihong Yu <zyu@yugabyte.com> wrote:
Hi,
+ * EXPLAIN ANALYZE CREATE TABLE AS or REFRESH MATERIALIZED VIEW + * WITH NO DATA is weird.Maybe it is clearer to spell out WITH NO DATA for both statements, instead of sharing it.
Done that way.
- if (!stmt->skipData) + if (!stmt->skipData && !explainInfo) ... + else if (explainInfo)It would be cleaner to put the 'if (explainInfo)' as the first check. That way, the check for skipData can be simplified.
Changed.
Thanks for review comments. Attaching v7 patch set with changes only
in 0002 patch. Please have a look.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v7-0001-Rearrange-Refresh-Mat-View-Code.patchapplication/octet-stream; name=v7-0001-Rearrange-Refresh-Mat-View-Code.patchDownload
From 26970ffd33e67324609a03f0f61eeb6406216a7f Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Fri, 5 Mar 2021 15:47:12 +0530
Subject: [PATCH v7 1/2] Rearrange Refresh Mat View Code
Currently, the function ExecRefreshMatView in matview.c is having
many lines of code which is not at all good from readability and
maintainability perspectives. This patch adds few functions and
moves the code from ExecRefreshMatView to them making the code
look better.
---
src/backend/commands/matview.c | 449 ++++++++++++++++++++-------------
1 file changed, 269 insertions(+), 180 deletions(-)
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..18e18fa627 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -64,7 +64,7 @@ static void transientrel_startup(DestReceiver *self, int operation, TupleDesc ty
static bool transientrel_receive(TupleTableSlot *slot, DestReceiver *self);
static void transientrel_shutdown(DestReceiver *self);
static void transientrel_destroy(DestReceiver *self);
-static uint64 refresh_matview_datafill(DestReceiver *dest, Query *query,
+static uint64 refresh_matview_datafill(Oid OIDNewHeap, Query *query,
const char *queryString);
static char *make_temptable_name_n(char *tempname, int n);
static void refresh_by_match_merge(Oid matviewOid, Oid tempOid, Oid relowner,
@@ -73,6 +73,16 @@ static void refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersist
static bool is_usable_unique_index(Relation indexRel);
static void OpenMatViewIncrementalMaintenance(void);
static void CloseMatViewIncrementalMaintenance(void);
+static Query *get_matview_query(RefreshMatViewStmt *stmt, Relation *rel,
+ Oid *objectId);
+static Query *rewrite_refresh_matview_query(Query *dataQuery);
+static Oid get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, char *relpersistence);
+static void match_matview_with_new_data(RefreshMatViewStmt *stmt,
+ Relation matviewRel, Oid matviewOid,
+ Oid OIDNewHeap, Oid relowner,
+ int save_sec_context,
+ char relpersistence, uint64 processed);
/*
* SetMatViewPopulatedState
@@ -140,127 +150,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
{
Oid matviewOid;
Relation matviewRel;
- RewriteRule *rule;
- List *actions;
Query *dataQuery;
- Oid tableSpace;
- Oid relowner;
Oid OIDNewHeap;
- DestReceiver *dest;
uint64 processed = 0;
- bool concurrent;
- LOCKMODE lockmode;
+ Oid relowner;
char relpersistence;
Oid save_userid;
int save_sec_context;
int save_nestlevel;
ObjectAddress address;
- /* Determine strength of lock needed. */
- concurrent = stmt->concurrent;
- lockmode = concurrent ? ExclusiveLock : AccessExclusiveLock;
-
- /*
- * Get a lock until end of transaction.
- */
- matviewOid = RangeVarGetRelidExtended(stmt->relation,
- lockmode, 0,
- RangeVarCallbackOwnsTable, NULL);
- matviewRel = table_open(matviewOid, NoLock);
-
- /* Make sure it is a materialized view. */
- if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("\"%s\" is not a materialized view",
- RelationGetRelationName(matviewRel))));
-
- /* Check that CONCURRENTLY is not specified if not populated. */
- if (concurrent && !RelationIsPopulated(matviewRel))
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
-
- /* Check that conflicting options have not been specified. */
- if (concurrent && stmt->skipData)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
-
- /*
- * Check that everything is correct for a refresh. Problems at this point
- * are internal errors, so elog is sufficient.
- */
- if (matviewRel->rd_rel->relhasrules == false ||
- matviewRel->rd_rules->numLocks < 1)
- elog(ERROR,
- "materialized view \"%s\" is missing rewrite information",
- RelationGetRelationName(matviewRel));
-
- if (matviewRel->rd_rules->numLocks > 1)
- elog(ERROR,
- "materialized view \"%s\" has too many rules",
- RelationGetRelationName(matviewRel));
-
- rule = matviewRel->rd_rules->rules[0];
- if (rule->event != CMD_SELECT || !(rule->isInstead))
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
- RelationGetRelationName(matviewRel));
-
- actions = rule->actions;
- if (list_length(actions) != 1)
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a single action",
- RelationGetRelationName(matviewRel));
-
- /*
- * Check that there is a unique index with no WHERE clause on one or more
- * columns of the materialized view if CONCURRENTLY is specified.
- */
- if (concurrent)
- {
- List *indexoidlist = RelationGetIndexList(matviewRel);
- ListCell *indexoidscan;
- bool hasUniqueIndex = false;
-
- foreach(indexoidscan, indexoidlist)
- {
- Oid indexoid = lfirst_oid(indexoidscan);
- Relation indexRel;
-
- indexRel = index_open(indexoid, AccessShareLock);
- hasUniqueIndex = is_usable_unique_index(indexRel);
- index_close(indexRel, AccessShareLock);
- if (hasUniqueIndex)
- break;
- }
-
- list_free(indexoidlist);
-
- if (!hasUniqueIndex)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot refresh materialized view \"%s\" concurrently",
- quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
- RelationGetRelationName(matviewRel))),
- errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
- }
-
- /*
- * The stored query was rewritten at the time of the MV definition, but
- * has not been scribbled on by the planner.
- */
- dataQuery = linitial_node(Query, actions);
-
- /*
- * Check for active uses of the relation in the current transaction, such
- * as open scans.
- *
- * NB: We count on this to protect us against problems with refreshing the
- * data using TABLE_INSERT_FROZEN.
- */
- CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+ /* Get the data generating query. */
+ dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);
/*
* Tentatively mark the matview as populated or not (this will roll back
@@ -281,27 +182,8 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- /* Concurrent refresh builds new data in temp tablespace, and does diff. */
- if (concurrent)
- {
- tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
- relpersistence = RELPERSISTENCE_TEMP;
- }
- else
- {
- tableSpace = matviewRel->rd_rel->reltablespace;
- relpersistence = matviewRel->rd_rel->relpersistence;
- }
-
- /*
- * Create the transient table that will receive the regenerated data. Lock
- * it against access by any other process until commit (by which time it
- * will be gone).
- */
- OIDNewHeap = make_new_heap(matviewOid, tableSpace, relpersistence,
- ExclusiveLock);
- LockRelationOid(OIDNewHeap, AccessExclusiveLock);
- dest = CreateTransientRelDestReceiver(OIDNewHeap);
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -311,40 +193,16 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/* Generate the data, if wanted. */
if (!stmt->skipData)
- processed = refresh_matview_datafill(dest, dataQuery, queryString);
-
- /* Make the matview match the newly generated data. */
- if (concurrent)
{
- int old_depth = matview_maintenance_depth;
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
- PG_TRY();
- {
- refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
- save_sec_context);
- }
- PG_CATCH();
- {
- matview_maintenance_depth = old_depth;
- PG_RE_THROW();
- }
- PG_END_TRY();
- Assert(matview_maintenance_depth == old_depth);
+ processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
+ queryString);
}
- else
- {
- refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
- /*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
- */
- pgstat_count_truncate(matviewRel);
- if (!stmt->skipData)
- pgstat_count_heap_insert(matviewRel, processed);
- }
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
@@ -373,30 +231,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/*
* refresh_matview_datafill
*
- * Execute the given query, sending result rows to "dest" (which will
- * insert them into the target matview).
+ * Create dest receiver and execute the given query, sending result rows to the
+ * dest receiver which will insert them into the target materialized view.
*
* Returns number of rows inserted.
*/
static uint64
-refresh_matview_datafill(DestReceiver *dest, Query *query,
- const char *queryString)
+refresh_matview_datafill(Oid OIDNewHeap, Query *query, const char *queryString)
{
- List *rewritten;
PlannedStmt *plan;
QueryDesc *queryDesc;
- Query *copied_query;
uint64 processed;
-
- /* Lock and rewrite, using a copy to preserve the original query. */
- copied_query = copyObject(query);
- AcquireRewriteLocks(copied_query, true, false);
- rewritten = QueryRewrite(copied_query);
-
- /* SELECT should never rewrite to more or less than one SELECT query */
- if (list_length(rewritten) != 1)
- elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
- query = (Query *) linitial(rewritten);
+ DestReceiver *dest;
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
@@ -413,6 +259,8 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
PushCopiedSnapshot(GetActiveSnapshot());
UpdateActiveSnapshotCommandId();
+ dest = CreateTransientRelDestReceiver(OIDNewHeap);
+
/* Create a QueryDesc, redirecting output to our tuple receiver */
queryDesc = CreateQueryDesc(plan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
@@ -933,3 +781,244 @@ CloseMatViewIncrementalMaintenance(void)
matview_maintenance_depth--;
Assert(matview_maintenance_depth >= 0);
}
+
+/*
+ * get_matview_query
+ *
+ * Open the refresh materialized view relation, perform sanity checks and also
+ * get the associated data generating query from it.
+ *
+ * Note that the refresh materialized view relation is opened here, it has to
+ * be closed in the caller.
+ */
+static Query *
+get_matview_query(RefreshMatViewStmt *stmt, Relation *rel, Oid *objectId)
+{
+ Oid matviewOid;
+ Relation matviewRel;
+ RewriteRule *rule;
+ List *actions;
+ Query *dataQuery;
+ LOCKMODE lockmode;
+
+ /* Determine strength of lock needed. */
+ lockmode = stmt->concurrent ? ExclusiveLock : AccessExclusiveLock;
+
+ /* Get a lock until end of transaction. */
+ matviewOid = RangeVarGetRelidExtended(stmt->relation, lockmode, 0,
+ RangeVarCallbackOwnsTable, NULL);
+ matviewRel = table_open(matviewOid, NoLock);
+
+ /* Make sure it is a materialized view. */
+ if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("\"%s\" is not a materialized view",
+ RelationGetRelationName(matviewRel))));
+
+ /* Check that CONCURRENTLY is not specified if not populated. */
+ if (stmt->concurrent && !RelationIsPopulated(matviewRel))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
+
+ /* Check that conflicting options have not been specified. */
+ if (stmt->concurrent && stmt->skipData)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
+
+ /*
+ * Check that everything is correct for a refresh. Problems at this point
+ * are internal errors, so elog is sufficient.
+ */
+ if (matviewRel->rd_rel->relhasrules == false ||
+ matviewRel->rd_rules->numLocks < 1)
+ elog(ERROR,
+ "materialized view \"%s\" is missing rewrite information",
+ RelationGetRelationName(matviewRel));
+
+ if (matviewRel->rd_rules->numLocks > 1)
+ elog(ERROR,
+ "materialized view \"%s\" has too many rules",
+ RelationGetRelationName(matviewRel));
+
+ rule = matviewRel->rd_rules->rules[0];
+ if (rule->event != CMD_SELECT || !(rule->isInstead))
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
+ RelationGetRelationName(matviewRel));
+
+ actions = rule->actions;
+ if (list_length(actions) != 1)
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a single action",
+ RelationGetRelationName(matviewRel));
+
+ /*
+ * Check that there is a unique index with no WHERE clause on one or more
+ * columns of the materialized view if CONCURRENTLY is specified.
+ */
+ if (stmt->concurrent)
+ {
+ List *indexoidlist = RelationGetIndexList(matviewRel);
+ ListCell *indexoidscan;
+ bool hasUniqueIndex = false;
+
+ foreach(indexoidscan, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(indexoidscan);
+ Relation indexRel;
+
+ indexRel = index_open(indexoid, AccessShareLock);
+ hasUniqueIndex = is_usable_unique_index(indexRel);
+ index_close(indexRel, AccessShareLock);
+ if (hasUniqueIndex)
+ break;
+ }
+
+ list_free(indexoidlist);
+
+ if (!hasUniqueIndex)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot refresh materialized view \"%s\" concurrently",
+ quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
+ RelationGetRelationName(matviewRel))),
+ errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
+ }
+
+ /*
+ * The stored query was rewritten at the time of the MV definition, but
+ * has not been scribbled on by the planner.
+ */
+ dataQuery = linitial_node(Query, actions);
+
+ /*
+ * Check for active uses of the relation in the current transaction, such
+ * as open scans.
+ *
+ * NB: We count on this to protect us against problems with refreshing the
+ * data using TABLE_INSERT_FROZEN.
+ */
+ CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+
+ *rel = matviewRel;
+ *objectId = matviewOid;
+
+ return dataQuery;
+}
+
+/*
+ * get_new_heap_oid
+ *
+ * Create a new heap and return its oid to which the refresh materialized view
+ * data is inserted into.
+ */
+static Oid
+get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid matviewOid,
+ char *relpersistence)
+{
+ Oid OIDNewHeap;
+ Oid tableSpace;
+
+ /* Concurrent refresh builds new data in temp tablespace, and does diff. */
+ if (stmt->concurrent)
+ {
+ tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
+ *relpersistence = RELPERSISTENCE_TEMP;
+ }
+ else
+ {
+ tableSpace = matviewRel->rd_rel->reltablespace;
+ *relpersistence = matviewRel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Create the transient table that will receive the regenerated data. Lock
+ * it against access by any other process until commit (by which time it
+ * will be gone).
+ */
+ OIDNewHeap = make_new_heap(matviewOid, tableSpace, *relpersistence,
+ ExclusiveLock);
+ LockRelationOid(OIDNewHeap, AccessExclusiveLock);
+
+ return OIDNewHeap;
+}
+
+/*
+ * match_matview_with_new_data
+ *
+ * Arrange the materialized view newly generated data to match the existing
+ * data i.e merge in case of CONCURRENTLY otherwise perform heap swap and
+ * truncate the materialized view.
+ */
+static void
+match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, Oid OIDNewHeap, Oid relowner,
+ int save_sec_context, char relpersistence,
+ uint64 processed)
+{
+ /* Make the materialized view match the newly generated data. */
+ if (stmt->concurrent)
+ {
+ int old_depth = matview_maintenance_depth;
+
+ PG_TRY();
+ {
+ refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
+ save_sec_context);
+ }
+ PG_CATCH();
+ {
+ matview_maintenance_depth = old_depth;
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ Assert(matview_maintenance_depth == old_depth);
+ }
+ else
+ {
+ refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
+
+ /*
+ * Inform stats collector about our activity: basically, we truncated
+ * the materialized view and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the
+ * inserts and deletes it issues get counted by lower-level code.)
+ */
+ pgstat_count_truncate(matviewRel);
+ if (!stmt->skipData)
+ pgstat_count_heap_insert(matviewRel, processed);
+ }
+}
+
+/*
+ * rewrite_refresh_matview_query
+ *
+ * Rewrite the refresh materialized view data generating query.
+ *
+ * Work on the copied query to preserve the original query. Because the
+ * rewriter and planner tend to scribble on the input, we make a preliminary
+ * copy of the source querytree. This prevents problems in the case that
+ * REFRESH MATERIALIZED VIEW is in a portal or plpgsql function and is executed
+ * repeatedly. (See also the same hack in EXPLAIN and PREPARE.)
+ */
+static Query *
+rewrite_refresh_matview_query(Query *dataQuery)
+{
+ List *rewritten;
+ Query *copied_query;
+
+ /* Lock and rewrite, using a copy to preserve the original query. */
+ copied_query = copyObject(dataQuery);
+ AcquireRewriteLocks(copied_query, true, false);
+ rewritten = QueryRewrite(copied_query);
+
+ /* SELECT should never rewrite to more or less than one SELECT query */
+ if (list_length(rewritten) != 1)
+ elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
+ dataQuery = (Query *) linitial(rewritten);
+
+ return dataQuery;
+}
--
2.25.1
v7-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchapplication/octet-stream; name=v7-0002-EXPLAIN-EXPLAIN-ANALYZE-REFRESH-MATERIALIZED-VIEW.patchDownload
From 13a41f167abf78df684832aa9e748e2b433b7d0f Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Mon, 8 Mar 2021 09:54:49 +0530
Subject: [PATCH v7] EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
Currently, explain/explain analyze refresh materialized view(RMV)
is not allowed. We do plan the materialized view query before every
refresh. I propose to show the explain/explain analyze of the
select part of the materialized view. It will be useful for the
user to know what exactly is being planned and executed as part of
RMV. Please note that we already have explain/explain analyze
CTAS/Create Mat View(CMV), where we show the explain/explain analyze
of the select part. This proposal will do the same thing.
The behaviour can be like this:
EXPLAIN REFRESH MATERIALIZED VIEW mv1; --> will not refresh the mat
view, but shows the select part's plan of mat view.
EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW mv1; --> will refresh the
mat view and shows the select part's plan of mat view.
---
doc/src/sgml/ref/explain.sgml | 2 +
src/backend/commands/explain.c | 50 +++++++++++-----
src/backend/commands/matview.c | 36 +++++++++---
src/backend/commands/prepare.c | 3 +-
src/backend/tcop/utility.c | 3 +-
src/include/commands/explain.h | 30 +++++++++-
src/include/commands/matview.h | 8 ++-
src/test/regress/expected/matview.out | 83 +++++++++++++++++++++++++++
src/test/regress/sql/matview.sql | 27 +++++++++
9 files changed, 217 insertions(+), 25 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index c4512332a0..a301b6b53f 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -95,6 +95,8 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
<command>EXPLAIN ANALYZE</command> on an
<command>INSERT</command>, <command>UPDATE</command>,
<command>DELETE</command>, <command>CREATE TABLE AS</command>,
+ <command>CREATE MATERIALIZED VIEW</command>,
+ <command>REFRESH MATERIALIZED VIEW</command>,
or <command>EXECUTE</command> statement
without letting the command affect your data, use this approach:
<programlisting>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index afc45429ba..57f92e0806 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -17,6 +17,7 @@
#include "catalog/pg_type.h"
#include "commands/createas.h"
#include "commands/defrem.h"
+#include "commands/matview.h"
#include "commands/prepare.h"
#include "executor/nodeHash.h"
#include "foreign/fdwapi.h"
@@ -53,10 +54,6 @@ explain_get_index_name_hook_type explain_get_index_name_hook = NULL;
#define X_CLOSE_IMMEDIATE 2
#define X_NOWHITESPACE 4
-static void ExplainOneQuery(Query *query, int cursorOptions,
- IntoClause *into, ExplainState *es,
- const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv);
static void ExplainPrintJIT(ExplainState *es, int jit_flags,
JitInstrumentation *ji);
static void report_triggers(ResultRelInfo *rInfo, bool show_relname,
@@ -274,7 +271,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
{
ExplainOneQuery(lfirst_node(Query, l),
CURSOR_OPT_PARALLEL_OK, NULL, es,
- pstate->p_sourcetext, params, pstate->p_queryEnv);
+ pstate->p_sourcetext, params, pstate->p_queryEnv,
+ NULL);
/* Separate plans with an appropriate separator */
if (lnext(rewritten, l) != NULL)
@@ -357,11 +355,11 @@ ExplainResultDesc(ExplainStmt *stmt)
*
* "into" is NULL unless we are explaining the contents of a CreateTableAsStmt.
*/
-static void
+void
ExplainOneQuery(Query *query, int cursorOptions,
IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
- QueryEnvironment *queryEnv)
+ QueryEnvironment *queryEnv, RefreshMatViewInfo *matviewInfo)
{
/* planner will not cope with utility statements */
if (query->commandType == CMD_UTILITY)
@@ -402,7 +400,8 @@ ExplainOneQuery(Query *query, int cursorOptions,
/* run it (if needed) and produce output */
ExplainOnePlan(plan, into, es, queryString, params, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ matviewInfo);
}
}
@@ -455,7 +454,7 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
CURSOR_OPT_PARALLEL_OK, ctas->into, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv, NULL);
}
else if (IsA(utilityStmt, DeclareCursorStmt))
{
@@ -474,7 +473,8 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
Assert(list_length(rewritten) == 1);
ExplainOneQuery(linitial_node(Query, rewritten),
dcs->options, NULL, es,
- queryString, params, queryEnv);
+ queryString, params, queryEnv,
+ NULL);
}
else if (IsA(utilityStmt, ExecuteStmt))
ExplainExecuteQuery((ExecuteStmt *) utilityStmt, into, es,
@@ -486,6 +486,16 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
else
ExplainDummyGroup("Notify", NULL, es);
}
+ else if(IsA(utilityStmt, RefreshMatViewStmt))
+ {
+ RefreshMatViewExplainInfo explainInfo;
+
+ explainInfo.es = es;
+ explainInfo.queryEnv = queryEnv;
+
+ ExecRefreshMatView((RefreshMatViewStmt *) utilityStmt,
+ queryString, params, NULL, &explainInfo);
+ }
else
{
if (es->format == EXPLAIN_FORMAT_TEXT)
@@ -512,7 +522,7 @@ void
ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
const char *queryString, ParamListInfo params,
QueryEnvironment *queryEnv, const instr_time *planduration,
- const BufferUsage *bufusage)
+ const BufferUsage *bufusage, RefreshMatViewInfo *matviewInfo)
{
DestReceiver *dest;
QueryDesc *queryDesc;
@@ -553,6 +563,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
*/
if (into)
dest = CreateIntoRelDestReceiver(into);
+ else if (matviewInfo && OidIsValid(matviewInfo->OIDNewHeap))
+ dest = CreateTransientRelDestReceiver(matviewInfo->OIDNewHeap);
else
dest = None_Receiver;
@@ -577,8 +589,12 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
{
ScanDirection dir;
- /* EXPLAIN ANALYZE CREATE TABLE AS WITH NO DATA is weird */
- if (into && into->skipData)
+ /*
+ * EXPLAIN ANALYZE CREATE TABLE AS WITH NO DATA or EXPLAN ANALYZE
+ * REFRESH MATERIALIZED VIEW WITH NO DATA is weird.
+ */
+ if ((into && into->skipData) ||
+ (matviewInfo && matviewInfo->skipData))
dir = NoMovementScanDirection;
else
dir = ForwardScanDirection;
@@ -586,6 +602,14 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
/* run the plan */
ExecutorRun(queryDesc, dir, 0L, true);
+ /*
+ * Collect the number of rows inserted in case of REFRESH MATERIALIZED
+ * VIEW which will be used while merging the newly generated data with
+ * the existing materialized view data in ExecRefreshMatView.
+ */
+ if (matviewInfo)
+ matviewInfo->processed = queryDesc->estate->es_processed;
+
/* run cleanup too */
ExecutorFinish(queryDesc);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 18e18fa627..c7d8c52712 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -146,7 +146,8 @@ SetMatViewPopulatedState(Relation relation, bool newstate)
*/
ObjectAddress
ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc)
+ ParamListInfo params, QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo)
{
Oid matviewOid;
Relation matviewRel;
@@ -182,8 +183,11 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
- &relpersistence);
+ if (explainInfo && !explainInfo->es->analyze)
+ OIDNewHeap = InvalidOid;
+ else
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -192,7 +196,24 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_RESTRICTED_OPERATION);
/* Generate the data, if wanted. */
- if (!stmt->skipData)
+ if (explainInfo)
+ {
+ RefreshMatViewInfo matViewInfo;
+
+ matViewInfo.OIDNewHeap = OIDNewHeap;
+ matViewInfo.skipData = stmt->skipData;
+ matViewInfo.processed = 0;
+
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
+
+ ExplainOneQuery(dataQuery,
+ CURSOR_OPT_PARALLEL_OK, NULL, explainInfo->es,
+ queryString, params, explainInfo->queryEnv,
+ &matViewInfo);
+
+ processed = matViewInfo.processed;
+ }
+ else if (!stmt->skipData)
{
dataQuery = rewrite_refresh_matview_query(dataQuery);
@@ -200,9 +221,10 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
queryString);
}
- match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
- relowner, save_sec_context, relpersistence,
- processed);
+ if (OidIsValid(OIDNewHeap))
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index f767751c71..56c7432879 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -673,7 +673,8 @@ ExplainExecuteQuery(ExecuteStmt *execstmt, IntoClause *into, ExplainState *es,
if (pstmt->commandType != CMD_UTILITY)
ExplainOnePlan(pstmt, into, es, query_string, paramLI, queryEnv,
- &planduration, (es->buffers ? &bufusage : NULL));
+ &planduration, (es->buffers ? &bufusage : NULL),
+ NULL);
else
ExplainOneUtility(pstmt->utilityStmt, into, es, query_string,
paramLI, queryEnv);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 05bb698cf4..20087addb4 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1621,7 +1621,8 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
address = ExecRefreshMatView((RefreshMatViewStmt *) parsetree,
- queryString, params, qc);
+ queryString, params, qc,
+ NULL);
}
PG_FINALLY();
{
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index e94d9e49cf..7f27d92df1 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -61,6 +61,27 @@ typedef struct ExplainState
ExplainWorkersState *workers_state; /* needed if parallel plan */
} ExplainState;
+/*
+ * Refresh Materialized View information passed across functions for EXPLAIN
+ * execution.
+ */
+typedef struct RefreshMatViewInfo
+{
+ /* Oid of the new heap created. */
+ Oid OIDNewHeap;
+ /* Is WITH NO DATA clause specified? */
+ bool skipData;
+ /* Number of rows inserted. */
+ uint64 processed;
+} RefreshMatViewInfo;
+
+/* EXPLAIN information shared to ExecRefreshMatView(). */
+typedef struct RefreshMatViewExplainInfo
+{
+ ExplainState *es;
+ QueryEnvironment *queryEnv;
+} RefreshMatViewExplainInfo;
+
/* Hook for plugins to get control in ExplainOneQuery() */
typedef void (*ExplainOneQuery_hook_type) (Query *query,
int cursorOptions,
@@ -91,7 +112,14 @@ extern void ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into,
ExplainState *es, const char *queryString,
ParamListInfo params, QueryEnvironment *queryEnv,
const instr_time *planduration,
- const BufferUsage *bufusage);
+ const BufferUsage *bufusage,
+ RefreshMatViewInfo *matviewInfo);
+
+extern void ExplainOneQuery(Query *query, int cursorOptions,
+ IntoClause *into, ExplainState *es,
+ const char *queryString, ParamListInfo params,
+ QueryEnvironment *queryEnv,
+ RefreshMatViewInfo *matviewInfo);
extern void ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc);
extern void ExplainPrintTriggers(ExplainState *es, QueryDesc *queryDesc);
diff --git a/src/include/commands/matview.h b/src/include/commands/matview.h
index 214b1c1df6..1d60180ebc 100644
--- a/src/include/commands/matview.h
+++ b/src/include/commands/matview.h
@@ -15,6 +15,7 @@
#define MATVIEW_H
#include "catalog/objectaddress.h"
+#include "commands/explain.h"
#include "nodes/params.h"
#include "nodes/parsenodes.h"
#include "tcop/dest.h"
@@ -23,8 +24,11 @@
extern void SetMatViewPopulatedState(Relation relation, bool newstate);
-extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
- ParamListInfo params, QueryCompletion *qc);
+extern ObjectAddress ExecRefreshMatView(RefreshMatViewStmt *stmt,
+ const char *queryString,
+ ParamListInfo params,
+ QueryCompletion *qc,
+ RefreshMatViewExplainInfo *explainInfo);
extern DestReceiver *CreateTransientRelDestReceiver(Oid oid);
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..305e511fea 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -668,3 +668,86 @@ NOTICE: relation "matview_ine_tab" already exists, skipping
(0 rows)
DROP MATERIALIZED VIEW matview_ine_tab;
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+ QUERY PLAN
+-----------------------------------------
+ Seq Scan on mv_exp_tbl (never executed)
+ Filter: (a > 5)
+(2 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+ERROR: materialized view "mv_exp" has not been populated
+HINT: Use the REFRESH MATERIALIZED VIEW command.
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------
+ Seq Scan on mv_exp_tbl
+ Filter: (a > 5)
+(2 rows)
+
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+ QUERY PLAN
+------------------------------------------------
+ Seq Scan on mv_exp_tbl (actual rows=5 loops=1)
+ Filter: (a > 5)
+ Rows Removed by Filter: 5
+(3 rows)
+
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+ a
+----
+ 6
+ 7
+ 8
+ 9
+ 10
+(5 rows)
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
diff --git a/src/test/regress/sql/matview.sql b/src/test/regress/sql/matview.sql
index 4a4bd0d6b6..67a45bbde7 100644
--- a/src/test/regress/sql/matview.sql
+++ b/src/test/regress/sql/matview.sql
@@ -287,3 +287,30 @@ EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
CREATE MATERIALIZED VIEW IF NOT EXISTS matview_ine_tab AS
SELECT 1 / 0 WITH NO DATA; -- ok
DROP MATERIALIZED VIEW matview_ine_tab;
+
+-- test cases for explain/explain analyze refresh materialized view
+CREATE TABLE mv_exp_tbl (a) AS SELECT * FROM generate_series(1, 10);
+CREATE MATERIALIZED VIEW mv_exp (a) AS
+ SELECT * FROM mv_exp_tbl WHERE a > 5;
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp WITH NO DATA;
+SELECT * FROM mv_exp ORDER BY 1; -- ERROR
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+REFRESH MATERIALIZED VIEW mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+CREATE UNIQUE INDEX ON mv_exp (a);
+EXPLAIN (COSTS OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
+ REFRESH MATERIALIZED VIEW CONCURRENTLY mv_exp;
+SELECT * FROM mv_exp ORDER BY 1; -- OK
+
+DROP MATERIALIZED VIEW mv_exp;
+DROP TABLE mv_exp_tbl;
--
2.25.1
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Mon, 08 Mar 2021 at 12:28, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
On Sun, Mar 7, 2021 at 10:13 PM Zhihong Yu <zyu@yugabyte.com> wrote:
Hi,
+ * EXPLAIN ANALYZE CREATE TABLE AS or REFRESH MATERIALIZED VIEW + * WITH NO DATA is weird.Maybe it is clearer to spell out WITH NO DATA for both statements, instead of sharing it.
Done that way.
- if (!stmt->skipData) + if (!stmt->skipData && !explainInfo) ... + else if (explainInfo)It would be cleaner to put the 'if (explainInfo)' as the first check. That way, the check for skipData can be simplified.
Changed.
Thanks for review comments. Attaching v7 patch set with changes only
in 0002 patch. Please have a look.
The v7 patch looks good to me, and there is no other advice, so I change
the status to "Ready for Committer".
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Sat, Mar 13, 2021 at 7:00 AM Japin Li <japinli@hotmail.com> wrote:
On Mon, 08 Mar 2021 at 12:28, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
On Sun, Mar 7, 2021 at 10:13 PM Zhihong Yu <zyu@yugabyte.com> wrote:
Hi,
+ * EXPLAIN ANALYZE CREATE TABLE AS or REFRESH MATERIALIZED VIEW + * WITH NO DATA is weird.Maybe it is clearer to spell out WITH NO DATA for both statements, instead of sharing it.
Done that way.
- if (!stmt->skipData) + if (!stmt->skipData && !explainInfo) ... + else if (explainInfo)It would be cleaner to put the 'if (explainInfo)' as the first check. That way, the check for skipData can be simplified.
Changed.
Thanks for review comments. Attaching v7 patch set with changes only
in 0002 patch. Please have a look.The v7 patch looks good to me, and there is no other advice, so I change
the status to "Ready for Committer".
Thanks for the review.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
[ Sorry for not looking at this thread sooner ]
Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> writes:
Currently, $subject is not allowed. We do plan the mat view query
before every refresh. I propose to show the explain/explain analyze of
the select part of the mat view in case of Refresh Mat View(RMV).
TBH, I think we should reject this. The problem with it is that it
binds us to the assumption that REFRESH MATERIALIZED VIEW has an
explainable plan. There are various people poking at ideas like
incremental matview updates, which might rely on some implementation
that doesn't exactly equate to a SQL query. Incremental updates are
hard enough already; they'll be even harder if they also have to
maintain compatibility with a pre-existing EXPLAIN behavior.
I don't really see that this feature buys us anything you can't
get by explaining the view's query, so I think we're better advised
to keep our options open about how REFRESH might be implemented
in future.
regards, tom lane
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Tue, Mar 16, 2021 at 1:15 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
[ Sorry for not looking at this thread sooner ]
Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> writes:
Currently, $subject is not allowed. We do plan the mat view query
before every refresh. I propose to show the explain/explain analyze of
the select part of the mat view in case of Refresh Mat View(RMV).TBH, I think we should reject this. The problem with it is that it
binds us to the assumption that REFRESH MATERIALIZED VIEW has an
explainable plan. There are various people poking at ideas like
incremental matview updates, which might rely on some implementation
that doesn't exactly equate to a SQL query. Incremental updates are
hard enough already; they'll be even harder if they also have to
maintain compatibility with a pre-existing EXPLAIN behavior.I don't really see that this feature buys us anything you can't
get by explaining the view's query, so I think we're better advised
to keep our options open about how REFRESH might be implemented
in future.
That makes sense to me. Thanks for the comments. I'm fine to withdraw the patch.
I would like to see if the 0001 patch(attaching here) will be useful
at all. It just splits up the existing ExecRefreshMatView into a few
functions to make the code readable. I'm okay to withdraw it if no one
agrees.
With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v7-0001-Rearrange-Refresh-Mat-View-Code.patchapplication/x-patch; name=v7-0001-Rearrange-Refresh-Mat-View-Code.patchDownload
From 26970ffd33e67324609a03f0f61eeb6406216a7f Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com>
Date: Fri, 5 Mar 2021 15:47:12 +0530
Subject: [PATCH v7 1/2] Rearrange Refresh Mat View Code
Currently, the function ExecRefreshMatView in matview.c is having
many lines of code which is not at all good from readability and
maintainability perspectives. This patch adds few functions and
moves the code from ExecRefreshMatView to them making the code
look better.
---
src/backend/commands/matview.c | 449 ++++++++++++++++++++-------------
1 file changed, 269 insertions(+), 180 deletions(-)
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..18e18fa627 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -64,7 +64,7 @@ static void transientrel_startup(DestReceiver *self, int operation, TupleDesc ty
static bool transientrel_receive(TupleTableSlot *slot, DestReceiver *self);
static void transientrel_shutdown(DestReceiver *self);
static void transientrel_destroy(DestReceiver *self);
-static uint64 refresh_matview_datafill(DestReceiver *dest, Query *query,
+static uint64 refresh_matview_datafill(Oid OIDNewHeap, Query *query,
const char *queryString);
static char *make_temptable_name_n(char *tempname, int n);
static void refresh_by_match_merge(Oid matviewOid, Oid tempOid, Oid relowner,
@@ -73,6 +73,16 @@ static void refresh_by_heap_swap(Oid matviewOid, Oid OIDNewHeap, char relpersist
static bool is_usable_unique_index(Relation indexRel);
static void OpenMatViewIncrementalMaintenance(void);
static void CloseMatViewIncrementalMaintenance(void);
+static Query *get_matview_query(RefreshMatViewStmt *stmt, Relation *rel,
+ Oid *objectId);
+static Query *rewrite_refresh_matview_query(Query *dataQuery);
+static Oid get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, char *relpersistence);
+static void match_matview_with_new_data(RefreshMatViewStmt *stmt,
+ Relation matviewRel, Oid matviewOid,
+ Oid OIDNewHeap, Oid relowner,
+ int save_sec_context,
+ char relpersistence, uint64 processed);
/*
* SetMatViewPopulatedState
@@ -140,127 +150,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
{
Oid matviewOid;
Relation matviewRel;
- RewriteRule *rule;
- List *actions;
Query *dataQuery;
- Oid tableSpace;
- Oid relowner;
Oid OIDNewHeap;
- DestReceiver *dest;
uint64 processed = 0;
- bool concurrent;
- LOCKMODE lockmode;
+ Oid relowner;
char relpersistence;
Oid save_userid;
int save_sec_context;
int save_nestlevel;
ObjectAddress address;
- /* Determine strength of lock needed. */
- concurrent = stmt->concurrent;
- lockmode = concurrent ? ExclusiveLock : AccessExclusiveLock;
-
- /*
- * Get a lock until end of transaction.
- */
- matviewOid = RangeVarGetRelidExtended(stmt->relation,
- lockmode, 0,
- RangeVarCallbackOwnsTable, NULL);
- matviewRel = table_open(matviewOid, NoLock);
-
- /* Make sure it is a materialized view. */
- if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("\"%s\" is not a materialized view",
- RelationGetRelationName(matviewRel))));
-
- /* Check that CONCURRENTLY is not specified if not populated. */
- if (concurrent && !RelationIsPopulated(matviewRel))
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
-
- /* Check that conflicting options have not been specified. */
- if (concurrent && stmt->skipData)
- ereport(ERROR,
- (errcode(ERRCODE_SYNTAX_ERROR),
- errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
-
- /*
- * Check that everything is correct for a refresh. Problems at this point
- * are internal errors, so elog is sufficient.
- */
- if (matviewRel->rd_rel->relhasrules == false ||
- matviewRel->rd_rules->numLocks < 1)
- elog(ERROR,
- "materialized view \"%s\" is missing rewrite information",
- RelationGetRelationName(matviewRel));
-
- if (matviewRel->rd_rules->numLocks > 1)
- elog(ERROR,
- "materialized view \"%s\" has too many rules",
- RelationGetRelationName(matviewRel));
-
- rule = matviewRel->rd_rules->rules[0];
- if (rule->event != CMD_SELECT || !(rule->isInstead))
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
- RelationGetRelationName(matviewRel));
-
- actions = rule->actions;
- if (list_length(actions) != 1)
- elog(ERROR,
- "the rule for materialized view \"%s\" is not a single action",
- RelationGetRelationName(matviewRel));
-
- /*
- * Check that there is a unique index with no WHERE clause on one or more
- * columns of the materialized view if CONCURRENTLY is specified.
- */
- if (concurrent)
- {
- List *indexoidlist = RelationGetIndexList(matviewRel);
- ListCell *indexoidscan;
- bool hasUniqueIndex = false;
-
- foreach(indexoidscan, indexoidlist)
- {
- Oid indexoid = lfirst_oid(indexoidscan);
- Relation indexRel;
-
- indexRel = index_open(indexoid, AccessShareLock);
- hasUniqueIndex = is_usable_unique_index(indexRel);
- index_close(indexRel, AccessShareLock);
- if (hasUniqueIndex)
- break;
- }
-
- list_free(indexoidlist);
-
- if (!hasUniqueIndex)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("cannot refresh materialized view \"%s\" concurrently",
- quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
- RelationGetRelationName(matviewRel))),
- errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
- }
-
- /*
- * The stored query was rewritten at the time of the MV definition, but
- * has not been scribbled on by the planner.
- */
- dataQuery = linitial_node(Query, actions);
-
- /*
- * Check for active uses of the relation in the current transaction, such
- * as open scans.
- *
- * NB: We count on this to protect us against problems with refreshing the
- * data using TABLE_INSERT_FROZEN.
- */
- CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+ /* Get the data generating query. */
+ dataQuery = get_matview_query(stmt, &matviewRel, &matviewOid);
/*
* Tentatively mark the matview as populated or not (this will roll back
@@ -281,27 +182,8 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
save_sec_context | SECURITY_LOCAL_USERID_CHANGE);
save_nestlevel = NewGUCNestLevel();
- /* Concurrent refresh builds new data in temp tablespace, and does diff. */
- if (concurrent)
- {
- tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
- relpersistence = RELPERSISTENCE_TEMP;
- }
- else
- {
- tableSpace = matviewRel->rd_rel->reltablespace;
- relpersistence = matviewRel->rd_rel->relpersistence;
- }
-
- /*
- * Create the transient table that will receive the regenerated data. Lock
- * it against access by any other process until commit (by which time it
- * will be gone).
- */
- OIDNewHeap = make_new_heap(matviewOid, tableSpace, relpersistence,
- ExclusiveLock);
- LockRelationOid(OIDNewHeap, AccessExclusiveLock);
- dest = CreateTransientRelDestReceiver(OIDNewHeap);
+ OIDNewHeap = get_new_heap_oid(stmt, matviewRel, matviewOid,
+ &relpersistence);
/*
* Now lock down security-restricted operations.
@@ -311,40 +193,16 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/* Generate the data, if wanted. */
if (!stmt->skipData)
- processed = refresh_matview_datafill(dest, dataQuery, queryString);
-
- /* Make the matview match the newly generated data. */
- if (concurrent)
{
- int old_depth = matview_maintenance_depth;
+ dataQuery = rewrite_refresh_matview_query(dataQuery);
- PG_TRY();
- {
- refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
- save_sec_context);
- }
- PG_CATCH();
- {
- matview_maintenance_depth = old_depth;
- PG_RE_THROW();
- }
- PG_END_TRY();
- Assert(matview_maintenance_depth == old_depth);
+ processed = refresh_matview_datafill(OIDNewHeap, dataQuery,
+ queryString);
}
- else
- {
- refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
- /*
- * Inform stats collector about our activity: basically, we truncated
- * the matview and inserted some new data. (The concurrent code path
- * above doesn't need to worry about this because the inserts and
- * deletes it issues get counted by lower-level code.)
- */
- pgstat_count_truncate(matviewRel);
- if (!stmt->skipData)
- pgstat_count_heap_insert(matviewRel, processed);
- }
+ match_matview_with_new_data(stmt, matviewRel, matviewOid, OIDNewHeap,
+ relowner, save_sec_context, relpersistence,
+ processed);
table_close(matviewRel, NoLock);
@@ -373,30 +231,18 @@ ExecRefreshMatView(RefreshMatViewStmt *stmt, const char *queryString,
/*
* refresh_matview_datafill
*
- * Execute the given query, sending result rows to "dest" (which will
- * insert them into the target matview).
+ * Create dest receiver and execute the given query, sending result rows to the
+ * dest receiver which will insert them into the target materialized view.
*
* Returns number of rows inserted.
*/
static uint64
-refresh_matview_datafill(DestReceiver *dest, Query *query,
- const char *queryString)
+refresh_matview_datafill(Oid OIDNewHeap, Query *query, const char *queryString)
{
- List *rewritten;
PlannedStmt *plan;
QueryDesc *queryDesc;
- Query *copied_query;
uint64 processed;
-
- /* Lock and rewrite, using a copy to preserve the original query. */
- copied_query = copyObject(query);
- AcquireRewriteLocks(copied_query, true, false);
- rewritten = QueryRewrite(copied_query);
-
- /* SELECT should never rewrite to more or less than one SELECT query */
- if (list_length(rewritten) != 1)
- elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
- query = (Query *) linitial(rewritten);
+ DestReceiver *dest;
/* Check for user-requested abort. */
CHECK_FOR_INTERRUPTS();
@@ -413,6 +259,8 @@ refresh_matview_datafill(DestReceiver *dest, Query *query,
PushCopiedSnapshot(GetActiveSnapshot());
UpdateActiveSnapshotCommandId();
+ dest = CreateTransientRelDestReceiver(OIDNewHeap);
+
/* Create a QueryDesc, redirecting output to our tuple receiver */
queryDesc = CreateQueryDesc(plan, queryString,
GetActiveSnapshot(), InvalidSnapshot,
@@ -933,3 +781,244 @@ CloseMatViewIncrementalMaintenance(void)
matview_maintenance_depth--;
Assert(matview_maintenance_depth >= 0);
}
+
+/*
+ * get_matview_query
+ *
+ * Open the refresh materialized view relation, perform sanity checks and also
+ * get the associated data generating query from it.
+ *
+ * Note that the refresh materialized view relation is opened here, it has to
+ * be closed in the caller.
+ */
+static Query *
+get_matview_query(RefreshMatViewStmt *stmt, Relation *rel, Oid *objectId)
+{
+ Oid matviewOid;
+ Relation matviewRel;
+ RewriteRule *rule;
+ List *actions;
+ Query *dataQuery;
+ LOCKMODE lockmode;
+
+ /* Determine strength of lock needed. */
+ lockmode = stmt->concurrent ? ExclusiveLock : AccessExclusiveLock;
+
+ /* Get a lock until end of transaction. */
+ matviewOid = RangeVarGetRelidExtended(stmt->relation, lockmode, 0,
+ RangeVarCallbackOwnsTable, NULL);
+ matviewRel = table_open(matviewOid, NoLock);
+
+ /* Make sure it is a materialized view. */
+ if (matviewRel->rd_rel->relkind != RELKIND_MATVIEW)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("\"%s\" is not a materialized view",
+ RelationGetRelationName(matviewRel))));
+
+ /* Check that CONCURRENTLY is not specified if not populated. */
+ if (stmt->concurrent && !RelationIsPopulated(matviewRel))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("CONCURRENTLY cannot be used when the materialized view is not populated")));
+
+ /* Check that conflicting options have not been specified. */
+ if (stmt->concurrent && stmt->skipData)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("CONCURRENTLY and WITH NO DATA options cannot be used together")));
+
+ /*
+ * Check that everything is correct for a refresh. Problems at this point
+ * are internal errors, so elog is sufficient.
+ */
+ if (matviewRel->rd_rel->relhasrules == false ||
+ matviewRel->rd_rules->numLocks < 1)
+ elog(ERROR,
+ "materialized view \"%s\" is missing rewrite information",
+ RelationGetRelationName(matviewRel));
+
+ if (matviewRel->rd_rules->numLocks > 1)
+ elog(ERROR,
+ "materialized view \"%s\" has too many rules",
+ RelationGetRelationName(matviewRel));
+
+ rule = matviewRel->rd_rules->rules[0];
+ if (rule->event != CMD_SELECT || !(rule->isInstead))
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a SELECT INSTEAD OF rule",
+ RelationGetRelationName(matviewRel));
+
+ actions = rule->actions;
+ if (list_length(actions) != 1)
+ elog(ERROR,
+ "the rule for materialized view \"%s\" is not a single action",
+ RelationGetRelationName(matviewRel));
+
+ /*
+ * Check that there is a unique index with no WHERE clause on one or more
+ * columns of the materialized view if CONCURRENTLY is specified.
+ */
+ if (stmt->concurrent)
+ {
+ List *indexoidlist = RelationGetIndexList(matviewRel);
+ ListCell *indexoidscan;
+ bool hasUniqueIndex = false;
+
+ foreach(indexoidscan, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(indexoidscan);
+ Relation indexRel;
+
+ indexRel = index_open(indexoid, AccessShareLock);
+ hasUniqueIndex = is_usable_unique_index(indexRel);
+ index_close(indexRel, AccessShareLock);
+ if (hasUniqueIndex)
+ break;
+ }
+
+ list_free(indexoidlist);
+
+ if (!hasUniqueIndex)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("cannot refresh materialized view \"%s\" concurrently",
+ quote_qualified_identifier(get_namespace_name(RelationGetNamespace(matviewRel)),
+ RelationGetRelationName(matviewRel))),
+ errhint("Create a unique index with no WHERE clause on one or more columns of the materialized view.")));
+ }
+
+ /*
+ * The stored query was rewritten at the time of the MV definition, but
+ * has not been scribbled on by the planner.
+ */
+ dataQuery = linitial_node(Query, actions);
+
+ /*
+ * Check for active uses of the relation in the current transaction, such
+ * as open scans.
+ *
+ * NB: We count on this to protect us against problems with refreshing the
+ * data using TABLE_INSERT_FROZEN.
+ */
+ CheckTableNotInUse(matviewRel, "REFRESH MATERIALIZED VIEW");
+
+ *rel = matviewRel;
+ *objectId = matviewOid;
+
+ return dataQuery;
+}
+
+/*
+ * get_new_heap_oid
+ *
+ * Create a new heap and return its oid to which the refresh materialized view
+ * data is inserted into.
+ */
+static Oid
+get_new_heap_oid(RefreshMatViewStmt *stmt, Relation matviewRel, Oid matviewOid,
+ char *relpersistence)
+{
+ Oid OIDNewHeap;
+ Oid tableSpace;
+
+ /* Concurrent refresh builds new data in temp tablespace, and does diff. */
+ if (stmt->concurrent)
+ {
+ tableSpace = GetDefaultTablespace(RELPERSISTENCE_TEMP, false);
+ *relpersistence = RELPERSISTENCE_TEMP;
+ }
+ else
+ {
+ tableSpace = matviewRel->rd_rel->reltablespace;
+ *relpersistence = matviewRel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Create the transient table that will receive the regenerated data. Lock
+ * it against access by any other process until commit (by which time it
+ * will be gone).
+ */
+ OIDNewHeap = make_new_heap(matviewOid, tableSpace, *relpersistence,
+ ExclusiveLock);
+ LockRelationOid(OIDNewHeap, AccessExclusiveLock);
+
+ return OIDNewHeap;
+}
+
+/*
+ * match_matview_with_new_data
+ *
+ * Arrange the materialized view newly generated data to match the existing
+ * data i.e merge in case of CONCURRENTLY otherwise perform heap swap and
+ * truncate the materialized view.
+ */
+static void
+match_matview_with_new_data(RefreshMatViewStmt *stmt, Relation matviewRel,
+ Oid matviewOid, Oid OIDNewHeap, Oid relowner,
+ int save_sec_context, char relpersistence,
+ uint64 processed)
+{
+ /* Make the materialized view match the newly generated data. */
+ if (stmt->concurrent)
+ {
+ int old_depth = matview_maintenance_depth;
+
+ PG_TRY();
+ {
+ refresh_by_match_merge(matviewOid, OIDNewHeap, relowner,
+ save_sec_context);
+ }
+ PG_CATCH();
+ {
+ matview_maintenance_depth = old_depth;
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ Assert(matview_maintenance_depth == old_depth);
+ }
+ else
+ {
+ refresh_by_heap_swap(matviewOid, OIDNewHeap, relpersistence);
+
+ /*
+ * Inform stats collector about our activity: basically, we truncated
+ * the materialized view and inserted some new data. (The concurrent
+ * code path above doesn't need to worry about this because the
+ * inserts and deletes it issues get counted by lower-level code.)
+ */
+ pgstat_count_truncate(matviewRel);
+ if (!stmt->skipData)
+ pgstat_count_heap_insert(matviewRel, processed);
+ }
+}
+
+/*
+ * rewrite_refresh_matview_query
+ *
+ * Rewrite the refresh materialized view data generating query.
+ *
+ * Work on the copied query to preserve the original query. Because the
+ * rewriter and planner tend to scribble on the input, we make a preliminary
+ * copy of the source querytree. This prevents problems in the case that
+ * REFRESH MATERIALIZED VIEW is in a portal or plpgsql function and is executed
+ * repeatedly. (See also the same hack in EXPLAIN and PREPARE.)
+ */
+static Query *
+rewrite_refresh_matview_query(Query *dataQuery)
+{
+ List *rewritten;
+ Query *copied_query;
+
+ /* Lock and rewrite, using a copy to preserve the original query. */
+ copied_query = copyObject(dataQuery);
+ AcquireRewriteLocks(copied_query, true, false);
+ rewritten = QueryRewrite(copied_query);
+
+ /* SELECT should never rewrite to more or less than one SELECT query */
+ if (list_length(rewritten) != 1)
+ elog(ERROR, "unexpected rewrite result for REFRESH MATERIALIZED VIEW");
+ dataQuery = (Query *) linitial(rewritten);
+
+ return dataQuery;
+}
--
2.25.1
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Tue, 16 Mar 2021 at 20:13, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
On Tue, Mar 16, 2021 at 1:15 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
[ Sorry for not looking at this thread sooner ]
Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> writes:
Currently, $subject is not allowed. We do plan the mat view query
before every refresh. I propose to show the explain/explain analyze of
the select part of the mat view in case of Refresh Mat View(RMV).TBH, I think we should reject this. The problem with it is that it
binds us to the assumption that REFRESH MATERIALIZED VIEW has an
explainable plan. There are various people poking at ideas like
incremental matview updates, which might rely on some implementation
that doesn't exactly equate to a SQL query. Incremental updates are
hard enough already; they'll be even harder if they also have to
maintain compatibility with a pre-existing EXPLAIN behavior.I don't really see that this feature buys us anything you can't
get by explaining the view's query, so I think we're better advised
to keep our options open about how REFRESH might be implemented
in future.That makes sense to me. Thanks for the comments. I'm fine to withdraw the patch.
I would like to see if the 0001 patch(attaching here) will be useful
at all. It just splits up the existing ExecRefreshMatView into a few
functions to make the code readable.
+1.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Tue, Mar 16, 2021 at 8:13 AM Bharath Rupireddy <
bharath.rupireddyforpostgres@gmail.com> wrote:
On Tue, Mar 16, 2021 at 1:15 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I don't really see that this feature buys us anything you can't
get by explaining the view's query, so I think we're better advised
to keep our options open about how REFRESH might be implemented
in future.That makes sense to me. Thanks for the comments. I'm fine to withdraw the
patch.
I would like to see if the 0001 patch(attaching here) will be useful
at all. It just splits up the existing ExecRefreshMatView into a few
functions to make the code readable. I'm okay to withdraw it if no one
agrees.
Side note for future reference: While the feature named in the CF entry has
been rejected, the remaining 0001 patch currently proposed no longer
matches the title, or category. It is possible within the CF app, and
helpful, to rename the entry when the scope changes.
The proposed patch in the CF for incremental view maintenance [1]https://commitfest.postgresql.org/33/2138/ does some
refactoring of its own in implementing the feature. I don't think it makes
sense to commit a refactoring that conflicts with that, while not
necessarily making life easier for that feature. Incremental view
maintenance is highly desirable, so I don't want to put up unnecessary
roadblocks.
[1]: https://commitfest.postgresql.org/33/2138/
--
John Naylor
EDB: http://www.enterprisedb.com
Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW
On Sat, Jul 10, 2021 at 5:19 PM John Naylor
<john.naylor@enterprisedb.com> wrote:
Side note for future reference: While the feature named in the CF entry has been rejected, the remaining 0001 patch currently proposed no longer matches the title, or category. It is possible within the CF app, and helpful, to rename the entry when the scope changes.
The proposed patch in the CF for incremental view maintenance [1] does some refactoring of its own in implementing the feature. I don't think it makes sense to commit a refactoring that conflicts with that, while not necessarily making life easier for that feature. Incremental view maintenance is highly desirable, so I don't want to put up unnecessary roadblocks.
Thanks. I'm okay to close the CF
entry(https://commitfest.postgresql.org/33/2928/) and stop this
thread.
Regards,
Bharath Rupireddy.