Backpatch b61d161c14
I propose to backpatch b61d161c14 [1]commit b61d161c146328ae6ba9ed937862d66e5c8b035a Author: Amit Kapila <akapila@postgresql.org> Date: Mon Mar 30 07:33:38 2020 +0530 (Introduce vacuum errcontext to
display additional information.). In the recent past, we have seen an
error report similar to "ERROR: found xmin 2157740646 from before
relfrozenxid 1197" from multiple EDB customers. A similar report is
seen on pgsql-bugs as well [2]/messages/by-id/20190807235154.erbmr4o4bo6vgnjv@alap3.anarazel.de which I think has triggered the
implementation of this feature for v13. Such reports mostly indicate
database corruption rather than any postgres bug which is also
indicated by the error-code (from before relfrozenxid) for this
message. I think there is a good reason to back-patch this as multiple
users are facing similar issues. This patch won't fix this issue but
it will help us in detecting the problematic part of the heap/index
and then if users wish they can delete the portion of data that
appeared to be corrupted and resume the operations on that relation.
I have tried to back-patch this for v12 and attached is the result.
The attached patch passes make check-world but I have yet to test it
manually and also prepare the patch for other branches once we agree
on this proposal.
Thoughts?
[1]: commit b61d161c146328ae6ba9ed937862d66e5c8b035a Author: Amit Kapila <akapila@postgresql.org> Date: Mon Mar 30 07:33:38 2020 +0530
commit b61d161c146328ae6ba9ed937862d66e5c8b035a
Author: Amit Kapila <akapila@postgresql.org>
Date: Mon Mar 30 07:33:38 2020 +0530
Introduce vacuum errcontext to display additional information.
The additional information displayed will be block number for error
occurring while processing heap and index name for error occurring
while processing the index.
[2]: /messages/by-id/20190807235154.erbmr4o4bo6vgnjv@alap3.anarazel.de
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
Introduce-vacuum-errcontext-to-display-additional-in.v12.patchapplication/octet-stream; name=Introduce-vacuum-errcontext-to-display-additional-in.v12.patchDownload
From 772103d67b8b518541209b0460f80e3342a406e6 Mon Sep 17 00:00:00 2001
From: Amit Kapila <akapila@postgresql.org>
Date: Mon, 22 Jun 2020 10:05:21 +0530
Subject: [PATCH] Introduce vacuum errcontext to display additional
information.
The additional information displayed will be block number for error
occurring while processing heap and index name for error occurring
while processing the index.
This will help us in diagnosing the problems that occur during a vacuum.
For ex. due to corruption (either caused by bad hardware or by some bug)
if we get some error while vacuuming, it can help us identify the block
in heap and or additional index information.
It sets up an error context callback to display additional information
with the error. During different phases of vacuum (heap scan, heap
vacuum, index vacuum, index clean up, heap truncate), we update the error
context callback to display appropriate information. We can extend it to
a bit more granular level like adding the phases for FSM operations or for
prefetching the blocks while truncating. However, I felt that it requires
adding many more error callback function calls and can make the code a bit
complex, so left those for now.
---
src/backend/access/heap/vacuumlazy.c | 185 +++++++++++++++++++++++++++++++++--
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 178 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3d4719d..a3f8f0b 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -110,8 +110,21 @@
*/
#define PREFETCH_SIZE ((BlockNumber) 32)
+/* Phases of vacuum during which we report error context. */
+typedef enum
+{
+ VACUUM_ERRCB_PHASE_UNKNOWN,
+ VACUUM_ERRCB_PHASE_SCAN_HEAP,
+ VACUUM_ERRCB_PHASE_VACUUM_INDEX,
+ VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
+ VACUUM_ERRCB_PHASE_TRUNCATE
+} VacErrPhase;
+
typedef struct LVRelStats
{
+ char *relnamespace;
+ char *relname;
/* useindex = true means two-pass strategy; false means one-pass */
bool useindex;
/* Overall statistics about rel */
@@ -136,6 +149,11 @@ typedef struct LVRelStats
int num_index_scans;
TransactionId latestRemovedXid;
bool lock_waiter_detected;
+
+ /* Used for error callback */
+ char *indname;
+ BlockNumber blkno; /* used only for heap operations */
+ VacErrPhase phase;
} LVRelStats;
@@ -175,6 +193,9 @@ static bool lazy_tid_reaped(ItemPointer itemptr, void *state);
static int vac_cmp_itemptr(const void *left, const void *right);
static bool heap_page_is_all_visible(Relation rel, Buffer buf,
TransactionId *visibility_cutoff_xid, bool *all_frozen);
+static void vacuum_error_callback(void *arg);
+static void update_vacuum_error_info(LVRelStats *errinfo, int phase,
+ BlockNumber blkno, char *indname);
/*
@@ -208,6 +229,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
double new_live_tuples;
TransactionId new_frozen_xid;
MultiXactId new_min_multi;
+ ErrorContextCallback errcallback;
Assert(params != NULL);
Assert(params->index_cleanup != VACOPT_TERNARY_DEFAULT);
@@ -257,6 +279,10 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
vacrelstats = (LVRelStats *) palloc0(sizeof(LVRelStats));
+ vacrelstats->relnamespace = get_namespace_name(RelationGetNamespace(onerel));
+ vacrelstats->relname = pstrdup(RelationGetRelationName(onerel));
+ vacrelstats->indname = NULL;
+ vacrelstats->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrelstats->old_rel_pages = onerel->rd_rel->relpages;
vacrelstats->old_live_tuples = onerel->rd_rel->reltuples;
vacrelstats->num_index_scans = 0;
@@ -268,6 +294,22 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
vacrelstats->useindex = (nindexes > 0 &&
params->index_cleanup == VACOPT_TERNARY_ENABLED);
+ /*
+ * Setup error traceback support for ereport(). The idea is to set up an
+ * error context callback to display additional information on any error
+ * during a vacuum. During different phases of vacuum (heap scan, heap
+ * vacuum, index vacuum, index clean up, heap truncate), we update the
+ * error context callback to display appropriate information.
+ *
+ * Note that the index vacuum and heap vacuum phases may be called
+ * multiple times in the middle of the heap scan phase. So the old phase
+ * information is restored at the end of those phases.
+ */
+ errcallback.callback = vacuum_error_callback;
+ errcallback.arg = vacrelstats;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
/* Do the vacuuming */
lazy_scan_heap(onerel, params, vacrelstats, Irel, nindexes, aggressive);
@@ -294,7 +336,19 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
* Optionally truncate the relation.
*/
if (should_attempt_truncation(params, vacrelstats))
+ {
+ /*
+ * Update error traceback information. This is the last phase during
+ * which we add context information to errors, so we don't need to
+ * revert to the previous phase.
+ */
+ update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_TRUNCATE,
+ vacrelstats->nonempty_pages, NULL);
lazy_truncate_heap(onerel, vacrelstats);
+ }
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
/* Report that we are now doing final cleanup */
pgstat_progress_update_param(PROGRESS_VACUUM_PHASE,
@@ -483,7 +537,6 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
BlockNumber nblocks,
blkno;
HeapTupleData tuple;
- char *relname;
TransactionId relfrozenxid = onerel->rd_rel->relfrozenxid;
TransactionId relminmxid = onerel->rd_rel->relminmxid;
BlockNumber empty_pages,
@@ -511,17 +564,16 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
pg_rusage_init(&ru0);
- relname = RelationGetRelationName(onerel);
if (aggressive)
ereport(elevel,
(errmsg("aggressively vacuuming \"%s.%s\"",
- get_namespace_name(RelationGetNamespace(onerel)),
- relname)));
+ vacrelstats->relnamespace,
+ vacrelstats->relname)));
else
ereport(elevel,
(errmsg("vacuuming \"%s.%s\"",
- get_namespace_name(RelationGetNamespace(onerel)),
- relname)));
+ vacrelstats->relnamespace,
+ vacrelstats->relname)));
empty_pages = vacuumed_pages = 0;
next_fsm_block_to_vacuum = (BlockNumber) 0;
@@ -642,6 +694,9 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_SCANNED, blkno);
+ update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_SCAN_HEAP,
+ blkno, NULL);
+
if (blkno == next_unskippable_block)
{
/* Time to advance next_unskippable_block */
@@ -1309,7 +1364,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
&& VM_ALL_VISIBLE(onerel, blkno, &vmbuffer))
{
elog(WARNING, "page is not marked all-visible but visibility map bit is set in relation \"%s\" page %u",
- relname, blkno);
+ vacrelstats->relname, blkno);
visibilitymap_clear(onerel, blkno, vmbuffer,
VISIBILITYMAP_VALID_BITS);
}
@@ -1330,7 +1385,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
else if (PageIsAllVisible(page) && has_dead_tuples)
{
elog(WARNING, "page containing dead tuples is marked as all-visible in relation \"%s\" page %u",
- relname, blkno);
+ vacrelstats->relname, blkno);
PageClearAllVisible(page);
MarkBufferDirty(buf);
visibilitymap_clear(onerel, blkno, vmbuffer,
@@ -1513,6 +1568,12 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
int npages;
PGRUsage ru0;
Buffer vmbuffer = InvalidBuffer;
+ LVRelStats olderrinfo;
+
+ /* Update error traceback information */
+ olderrinfo = *vacrelstats;
+ update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ InvalidBlockNumber, NULL);
pg_rusage_init(&ru0);
npages = 0;
@@ -1528,6 +1589,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
vacuum_delay_point();
tblk = ItemPointerGetBlockNumber(&vacrelstats->dead_tuples[tupindex]);
+ vacrelstats->blkno = tblk;
buf = ReadBufferExtended(onerel, MAIN_FORKNUM, tblk, RBM_NORMAL,
vac_strategy);
if (!ConditionalLockBufferForCleanup(buf))
@@ -1559,6 +1621,12 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
RelationGetRelationName(onerel),
tupindex, npages),
errdetail_internal("%s", pg_rusage_show(&ru0))));
+
+ /* Revert to the previous phase information for error traceback */
+ update_vacuum_error_info(vacrelstats,
+ olderrinfo.phase,
+ olderrinfo.blkno,
+ olderrinfo.indname);
}
/*
@@ -1580,9 +1648,15 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
+ LVRelStats olderrinfo;
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
+ /* Update error traceback information */
+ olderrinfo = *vacrelstats;
+ update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ blkno, NULL);
+
START_CRIT_SECTION();
for (; tupindex < vacrelstats->num_dead_tuples; tupindex++)
@@ -1659,6 +1733,11 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
*vmbuffer, visibility_cutoff_xid, flags);
}
+ /* Revert to the previous phase information for error traceback */
+ update_vacuum_error_info(vacrelstats,
+ olderrinfo.phase,
+ olderrinfo.blkno,
+ olderrinfo.indname);
return tupindex;
}
@@ -1729,6 +1808,7 @@ lazy_vacuum_index(Relation indrel,
{
IndexVacuumInfo ivinfo;
PGRUsage ru0;
+ LVRelStats olderrinfo;
pg_rusage_init(&ru0);
@@ -1741,6 +1821,13 @@ lazy_vacuum_index(Relation indrel,
ivinfo.num_heap_tuples = vacrelstats->old_live_tuples;
ivinfo.strategy = vac_strategy;
+ /* Update error traceback information */
+ olderrinfo = *vacrelstats;
+ update_vacuum_error_info(vacrelstats,
+ VACUUM_ERRCB_PHASE_VACUUM_INDEX,
+ InvalidBlockNumber,
+ RelationGetRelationName(indrel));
+
/* Do bulk deletion */
*stats = index_bulk_delete(&ivinfo, *stats,
lazy_tid_reaped, (void *) vacrelstats);
@@ -1750,6 +1837,12 @@ lazy_vacuum_index(Relation indrel,
RelationGetRelationName(indrel),
vacrelstats->num_dead_tuples),
errdetail_internal("%s", pg_rusage_show(&ru0))));
+
+ /* Revert to the previous phase information for error traceback */
+ update_vacuum_error_info(vacrelstats,
+ olderrinfo.phase,
+ olderrinfo.blkno,
+ olderrinfo.indname);
}
/*
@@ -1762,6 +1855,7 @@ lazy_cleanup_index(Relation indrel,
{
IndexVacuumInfo ivinfo;
PGRUsage ru0;
+ LVRelStats olderrcbarg;
pg_rusage_init(&ru0);
@@ -1779,8 +1873,21 @@ lazy_cleanup_index(Relation indrel,
ivinfo.num_heap_tuples = vacrelstats->new_rel_tuples;
ivinfo.strategy = vac_strategy;
+ /* Update error traceback information */
+ olderrcbarg = *vacrelstats;
+ update_vacuum_error_info(vacrelstats,
+ VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
+ InvalidBlockNumber,
+ RelationGetRelationName(indrel));
+
stats = index_vacuum_cleanup(&ivinfo, stats);
+ /* Revert back to the old phase information for error traceback */
+ update_vacuum_error_info(vacrelstats,
+ olderrcbarg.phase,
+ olderrcbarg.blkno,
+ olderrcbarg.indname);
+
if (!stats)
return;
@@ -1936,6 +2043,7 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
* were vacuuming.
*/
new_rel_pages = count_nondeletable_pages(onerel, vacrelstats);
+ vacrelstats->blkno = new_rel_pages;
if (new_rel_pages >= old_rel_pages)
{
@@ -2339,3 +2447,64 @@ heap_page_is_all_visible(Relation rel, Buffer buf,
return all_visible;
}
+
+/*
+ * Error context callback for errors occurring during vacuum.
+ */
+static void
+vacuum_error_callback(void *arg)
+{
+ LVRelStats *errinfo = arg;
+
+ switch (errinfo->phase)
+ {
+ case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if (BlockNumberIsValid(errinfo->blkno))
+ errcontext("while scanning block %u of relation \"%s.%s\"",
+ errinfo->blkno, errinfo->relnamespace, errinfo->relname);
+ break;
+
+ case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if (BlockNumberIsValid(errinfo->blkno))
+ errcontext("while vacuuming block %u of relation \"%s.%s\"",
+ errinfo->blkno, errinfo->relnamespace, errinfo->relname);
+ break;
+
+ case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
+ errinfo->indname, errinfo->relnamespace, errinfo->relname);
+ break;
+
+ case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
+ errinfo->indname, errinfo->relnamespace, errinfo->relname);
+ break;
+
+ case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if (BlockNumberIsValid(errinfo->blkno))
+ errcontext("while truncating relation \"%s.%s\" to %u blocks",
+ errinfo->relnamespace, errinfo->relname, errinfo->blkno);
+ break;
+
+ case VACUUM_ERRCB_PHASE_UNKNOWN:
+ default:
+ return; /* do nothing; the errinfo may not be
+ * initialized */
+ }
+}
+
+/* Update vacuum error callback for the current phase, block, and index. */
+static void
+update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno,
+ char *indname)
+{
+ errinfo->blkno = blkno;
+ errinfo->phase = phase;
+
+ /* Free index name from any previous phase */
+ if (errinfo->indname)
+ pfree(errinfo->indname);
+
+ /* For index phases, save the name of the current index for the callback */
+ errinfo->indname = indname ? pstrdup(indname) : NULL;
+}
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 2f23dc3..022a3d4 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2551,6 +2551,7 @@ UserMapping
UserOpts
VacAttrStats
VacAttrStatsP
+VacErrPhase
VacOptTernaryValue
VacuumParams
VacuumRelation
--
1.8.3.1
On Mon, Jun 22, 2020 at 10:35 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
I propose to backpatch b61d161c14 [1] (Introduce vacuum errcontext to
display additional information.). In the recent past, we have seen an
error report similar to "ERROR: found xmin 2157740646 from before
relfrozenxid 1197" from multiple EDB customers. A similar report is
seen on pgsql-bugs as well [2] which I think has triggered the
implementation of this feature for v13. Such reports mostly indicate
database corruption rather than any postgres bug which is also
indicated by the error-code (from before relfrozenxid) for this
message.
Sorry, the error-code I want to refer to in above sentence was
ERRCODE_DATA_CORRUPTED.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
On 2020-Jun-22, Amit Kapila wrote:
I propose to backpatch b61d161c14 [1] (Introduce vacuum errcontext to
display additional information.). In the recent past, we have seen an
error report similar to "ERROR: found xmin 2157740646 from before
relfrozenxid 1197" from multiple EDB customers. A similar report is
seen on pgsql-bugs as well [2] which I think has triggered the
implementation of this feature for v13.
+1 to backpatching this change. I did not review your actual patch,
though.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi,
On 2020-06-22 10:35:47 +0530, Amit Kapila wrote:
I propose to backpatch b61d161c14 [1] (Introduce vacuum errcontext to
display additional information.). In the recent past, we have seen an
error report similar to "ERROR: found xmin 2157740646 from before
relfrozenxid 1197" from multiple EDB customers. A similar report is
seen on pgsql-bugs as well [2] which I think has triggered the
implementation of this feature for v13. Such reports mostly indicate
database corruption rather than any postgres bug which is also
indicated by the error-code (from before relfrozenxid) for this
message. I think there is a good reason to back-patch this as multiple
users are facing similar issues. This patch won't fix this issue but
it will help us in detecting the problematic part of the heap/index
and then if users wish they can delete the portion of data that
appeared to be corrupted and resume the operations on that relation.I have tried to back-patch this for v12 and attached is the result.
The attached patch passes make check-world but I have yet to test it
manually and also prepare the patch for other branches once we agree
on this proposal.
I think having the additional information in the back branches would be
good. But on the other hand I think this is a somewhat large change
to backpatch, and it hasn't yet much real world exposure.
I'm also uncomfortable with the approach of just copying all of
LVRelStats in several places:
/*
@@ -1580,9 +1648,15 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
+ LVRelStats olderrinfo;pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
+ /* Update error traceback information */ + olderrinfo = *vacrelstats; + update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP, + blkno, NULL); + START_CRIT_SECTION();for (; tupindex < vacrelstats->num_dead_tuples; tupindex++)
@@ -1659,6 +1733,11 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
*vmbuffer, visibility_cutoff_xid, flags);
}+ /* Revert to the previous phase information for error traceback */ + update_vacuum_error_info(vacrelstats, + olderrinfo.phase, + olderrinfo.blkno, + olderrinfo.indname); return tupindex; }
To me that's a very weird approach. It's fragile because we need to be
sure that there's no updates to the wrong LVRelStats for important
things, and it has a good bit of potential to be inefficient because
LVRelStats isn't exactly small. This pretty much relies on the compiler
doing good enough escape analysis to realize that most parts of
olderrinfo aren't touched.
Greetings,
Andres Freund
On Mon, Jun 22, 2020 at 01:09:39PM -0700, Andres Freund wrote:
On 2020-06-22 10:35:47 +0530, Amit Kapila wrote:
I propose to backpatch b61d161c14 [1] (Introduce vacuum errcontext to
display additional information.).
...
I think having the additional information in the back branches would be
good. But on the other hand I think this is a somewhat large change
to backpatch, and it hasn't yet much real world exposure.
I see that's nontrivial to cherry-pick due to parallel vacuum changes, and due
to re-arranging calls to pgstat_progress.
Since the next minor releases are in August, and PG13 expected to be released
~October, we could defer backpatching until November (or later).
I'm also uncomfortable with the approach of just copying all of
LVRelStats in several places:/*
@@ -1580,9 +1648,15 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
+ LVRelStats olderrinfo;
I guess the alternative is to write like
LVRelStats olderrinfo = {
.phase = vacrelstats.phase,
.blkno = vacrelstats.blkno,
.indname = vacrelstats.indname,
};
--
Justin
Hi,
On 2020-06-22 15:43:11 -0500, Justin Pryzby wrote:
On Mon, Jun 22, 2020 at 01:09:39PM -0700, Andres Freund wrote:
I'm also uncomfortable with the approach of just copying all of
LVRelStats in several places:/*
@@ -1580,9 +1648,15 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
+ LVRelStats olderrinfo;I guess the alternative is to write like
LVRelStats olderrinfo = {
.phase = vacrelstats.phase,
.blkno = vacrelstats.blkno,
.indname = vacrelstats.indname,
};
No, I don't think that's a solution. I think it's wrong to have
something like olderrinfo in the first place. Using a struct with ~25
members to store the current state of three variables just doesn't make
sense. Why isn't this just a LVSavedPosition struct or something like
that?
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
No, I don't think that's a solution. I think it's wrong to have
something like olderrinfo in the first place. Using a struct with ~25
members to store the current state of three variables just doesn't make
sense. Why isn't this just a LVSavedPosition struct or something like
that?
That seems like rather pointless micro-optimization really; the struct's
not *that* large. But I have a different complaint now that I look at
this code: is it safe at all? I see that the indname field is a pointer
to who-knows-where. If it's possible in the first place for that to
change while this code runs, then what guarantees that we won't be
restoring a dangling pointer to freed memory?
regards, tom lane
On Mon, Jun 22, 2020 at 06:15:27PM -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
No, I don't think that's a solution. I think it's wrong to have
something like olderrinfo in the first place. Using a struct with ~25
members to store the current state of three variables just doesn't make
sense. Why isn't this just a LVSavedPosition struct or something like
that?That seems like rather pointless micro-optimization really; the struct's
not *that* large. But I have a different complaint now that I look at
this code: is it safe at all? I see that the indname field is a pointer
to who-knows-where. If it's possible in the first place for that to
change while this code runs, then what guarantees that we won't be
restoring a dangling pointer to freed memory?
I'm not sure it addresses your concern, but we talked a bunch about safety
starting here:
/messages/by-id/20200326150457.GB17431@telsasoft.com
..and concluding with an explanation about CHECK_FOR_INTERRUPTS.
20200326150457.GB17431@telsasoft.com
|And I think you're right: we only save state when the calling function has a
|indname=NULL, so we never "put back" a non-NULL indname. We go from having a
|indname=NULL at lazy_scan_heap to not not-NULL at lazy_vacuum_index, and never
|the other way around. So once we've "reverted back", 1) the pointer is null;
|and, 2) the callback function doesn't access it for the previous/reverted phase
|anyway.
When this function is called by lazy_vacuum_{heap,page,index}, it's also called
a 2nd time to restore the previous phase information. When it's called the
first time by lazy_vacuum_index(), it does errinfo->indname = pstrdup(indname),
and on the 2nd call then does pfree(errinfo->indame), followed by
errinfo->indname = NULL.
|static void
|update_vacuum_error_info(LVSavedPosition *errinfo, int phase, BlockNumber blkno,
| char *indname)
|{
| errinfo->blkno = blkno;
| errinfo->phase = phase;
|
| /* Free index name from any previous phase */
| if (errinfo->indname)
| pfree(errinfo->indname);
|
| /* For index phases, save the name of the current index for the callback */
| errinfo->indname = indname ? pstrdup(indname) : NULL;
|}
If it's inadequately clear, maybe we should do:
if (errinfo->indname)
+ {
pfree(errinfo->indname);
+ Assert(indname == NULL);
+ }
--
Justin
Justin Pryzby <pryzby@telsasoft.com> writes:
On Mon, Jun 22, 2020 at 06:15:27PM -0400, Tom Lane wrote:
That seems like rather pointless micro-optimization really; the struct's
not *that* large. But I have a different complaint now that I look at
this code: is it safe at all? I see that the indname field is a pointer
to who-knows-where. If it's possible in the first place for that to
change while this code runs, then what guarantees that we won't be
restoring a dangling pointer to freed memory?
I'm not sure it addresses your concern, but we talked a bunch about safety
starting here:
/messages/by-id/20200326150457.GB17431@telsasoft.com
..and concluding with an explanation about CHECK_FOR_INTERRUPTS.
20200326150457.GB17431@telsasoft.com
|And I think you're right: we only save state when the calling function has a
|indname=NULL, so we never "put back" a non-NULL indname. We go from having a
|indname=NULL at lazy_scan_heap to not not-NULL at lazy_vacuum_index, and never
|the other way around. So once we've "reverted back", 1) the pointer is null;
|and, 2) the callback function doesn't access it for the previous/reverted phase
|anyway.
If we're relying on that, I'd replace the "save" action by an Assert that
indname is NULL, and the "restore" action by just assigning NULL again.
That eliminates all concern about whether the restored value is valid.
regards, tom lane
On Mon, Jun 22, 2020 at 01:57:12PM -0700, Andres Freund wrote:
On 2020-06-22 15:43:11 -0500, Justin Pryzby wrote:
On Mon, Jun 22, 2020 at 01:09:39PM -0700, Andres Freund wrote:
I'm also uncomfortable with the approach of just copying all of
LVRelStats in several places:/*
@@ -1580,9 +1648,15 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
+ LVRelStats olderrinfo;I guess the alternative is to write like
LVRelStats olderrinfo = {
.phase = vacrelstats.phase,
.blkno = vacrelstats.blkno,
.indname = vacrelstats.indname,
};No, I don't think that's a solution. I think it's wrong to have
something like olderrinfo in the first place. Using a struct with ~25
members to store the current state of three variables just doesn't make
sense. Why isn't this just a LVSavedPosition struct or something like
that?
I'd used LVRelStats on your suggestion:
/messages/by-id/20191211165425.4ewww2s5k5cafi4l@alap3.anarazel.de
/messages/by-id/20200120191305.sxi44cedhtxwr3ag@alap3.anarazel.de
I understood the goal to be avoiding the need to add a new struct, when most
functions are already passed LVRelStats *vacrelstats.
But maybe I misunderstood. (Also, back in January, the callback was only used
for scan-heap phase, so it's increased in scope several times).
Anyway, I put together some patches for discussion purposes.
--
Justin
Attachments:
0001-Rename-from-errcbarg.patchtext/x-diff; charset=us-asciiDownload
From 82daf888eb6834400be482affe38c1f23851826b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 22 Jun 2020 18:22:58 -0500
Subject: [PATCH 1/4] Rename from "errcbarg"..
..the name of which Alvaro didn't like. I renamed this, but it was
accidentally re-introduced while trading patches with Amit.
We should *also* rename VACUUM_ERRCB* to ERRINFO, but I didn't do that here.
---
src/backend/access/heap/vacuumlazy.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3bef0e124b..a6a5d906c0 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -2459,7 +2459,7 @@ lazy_cleanup_index(Relation indrel,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrcbarg;
+ LVRelStats olderrinfo;
pg_rusage_init(&ru0);
@@ -2473,7 +2473,7 @@ lazy_cleanup_index(Relation indrel,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrcbarg = *vacrelstats;
+ olderrinfo = *vacrelstats;
update_vacuum_error_info(vacrelstats,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber,
@@ -2483,9 +2483,9 @@ lazy_cleanup_index(Relation indrel,
/* Revert back to the old phase information for error traceback */
update_vacuum_error_info(vacrelstats,
- olderrcbarg.phase,
- olderrcbarg.blkno,
- olderrcbarg.indname);
+ olderrinfo.phase,
+ olderrinfo.blkno,
+ olderrinfo.indname);
if (!(*stats))
return;
--
2.17.0
0002-Add-assert-and-document-why-indname-is-safe.patchtext/x-diff; charset=us-asciiDownload
From 308faea3626e8c270eb75659bec3f7e15125a73a Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 22 Jun 2020 18:51:35 -0500
Subject: [PATCH 2/4] Add assert and document why indname is safe
---
src/backend/access/heap/vacuumlazy.c | 19 ++++++++++++++++---
1 file changed, 16 insertions(+), 3 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index a6a5d906c0..40173480b8 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -3606,10 +3606,23 @@ update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno,
errinfo->blkno = blkno;
errinfo->phase = phase;
- /* Free index name from any previous phase */
if (errinfo->indname)
+ {
+ /*
+ * indname is only ever saved during lazy_vacuum_index(), which
+ * during which the phase information is not not further
+ * manipulated, until it's restored before returning from
+ * lazy_vacuum_index().
+ */
+ Assert(indname == NULL);
+
pfree(errinfo->indname);
+ errinfo->indname = NULL;
+ }
- /* For index phases, save the name of the current index for the callback */
- errinfo->indname = indname ? pstrdup(indname) : NULL;
+ if (indname)
+ {
+ /* For index phases, save the name of the current index for the callback */
+ errinfo->indname = pstrdup(indname);
+ }
}
--
2.17.0
0003-Move-error-information-into-a-separate-struct.patchtext/x-diff; charset=us-asciiDownload
From 87ff6cc66510233b1b86a250e8ac48e52b7d54c2 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 22 Jun 2020 16:36:14 -0500
Subject: [PATCH 3/4] Move error information into a separate struct..
..to avoid copying large LVRelStats struct.
Andres things this is a problem but Tom thinks it's a micro-optimization.
See: b61d161c146328ae6ba9ed937862d66e5c8b035a
This also moves relname and relnamespace (see ef75140fe).
Discussion: https://www.postgresql.org/message-id/20200622200939.jkuniq3vtiumeszj%40alap3.anarazel.de
---
src/backend/access/heap/vacuumlazy.c | 116 ++++++++++++++-------------
1 file changed, 62 insertions(+), 54 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 40173480b8..6bd2867660 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -288,10 +288,17 @@ typedef struct LVParallelState
int nindexes_parallel_condcleanup;
} LVParallelState;
-typedef struct LVRelStats
+typedef struct
{
char *relnamespace;
char *relname;
+ char *indname;
+ BlockNumber blkno; /* used only for heap operations */
+ VacErrPhase phase;
+} LVSavedPosition;
+
+typedef struct LVRelStats
+{
/* useindex = true means two-pass strategy; false means one-pass */
bool useindex;
/* Overall statistics about rel */
@@ -313,10 +320,7 @@ typedef struct LVRelStats
TransactionId latestRemovedXid;
bool lock_waiter_detected;
- /* Used for error callback */
- char *indname;
- BlockNumber blkno; /* used only for heap operations */
- VacErrPhase phase;
+ LVSavedPosition errinfo; /* Used for error callback */
} LVRelStats;
/* A few variables that don't seem worth passing around as parameters */
@@ -388,7 +392,7 @@ static void end_parallel_vacuum(Relation *Irel, IndexBulkDeleteResult **stats,
static LVSharedIndStats *get_indstats(LVShared *lvshared, int n);
static bool skip_parallel_vacuum_index(Relation indrel, LVShared *lvshared);
static void vacuum_error_callback(void *arg);
-static void update_vacuum_error_info(LVRelStats *errinfo, int phase,
+static void update_vacuum_error_info(LVSavedPosition *errinfo, int phase,
BlockNumber blkno, char *indname);
@@ -475,10 +479,10 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
vacrelstats = (LVRelStats *) palloc0(sizeof(LVRelStats));
- vacrelstats->relnamespace = get_namespace_name(RelationGetNamespace(onerel));
- vacrelstats->relname = pstrdup(RelationGetRelationName(onerel));
- vacrelstats->indname = NULL;
- vacrelstats->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
+ vacrelstats->errinfo.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
+ vacrelstats->errinfo.relname = pstrdup(RelationGetRelationName(onerel));
+ vacrelstats->errinfo.indname = NULL;
+ vacrelstats->errinfo.phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrelstats->old_rel_pages = onerel->rd_rel->relpages;
vacrelstats->old_live_tuples = onerel->rd_rel->reltuples;
vacrelstats->num_index_scans = 0;
@@ -502,7 +506,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
* information is restored at the end of those phases.
*/
errcallback.callback = vacuum_error_callback;
- errcallback.arg = vacrelstats;
+ errcallback.arg = &vacrelstats->errinfo;
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
@@ -538,7 +542,8 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
* which we add context information to errors, so we don't need to
* revert to the previous phase.
*/
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_TRUNCATE,
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ VACUUM_ERRCB_PHASE_TRUNCATE,
vacrelstats->nonempty_pages, NULL);
lazy_truncate_heap(onerel, vacrelstats);
}
@@ -650,8 +655,8 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
}
appendStringInfo(&buf, msgfmt,
get_database_name(MyDatabaseId),
- vacrelstats->relnamespace,
- vacrelstats->relname,
+ vacrelstats->errinfo.relnamespace,
+ vacrelstats->errinfo.relname,
vacrelstats->num_index_scans);
appendStringInfo(&buf, _("pages: %u removed, %u remain, %u skipped due to pins, %u skipped frozen\n"),
vacrelstats->pages_removed,
@@ -786,13 +791,13 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
if (aggressive)
ereport(elevel,
(errmsg("aggressively vacuuming \"%s.%s\"",
- vacrelstats->relnamespace,
- vacrelstats->relname)));
+ vacrelstats->errinfo.relnamespace,
+ vacrelstats->errinfo.relname)));
else
ereport(elevel,
(errmsg("vacuuming \"%s.%s\"",
- vacrelstats->relnamespace,
- vacrelstats->relname)));
+ vacrelstats->errinfo.relnamespace,
+ vacrelstats->errinfo.relname)));
empty_pages = vacuumed_pages = 0;
next_fsm_block_to_vacuum = (BlockNumber) 0;
@@ -828,7 +833,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
if (params->nworkers > 0)
ereport(WARNING,
(errmsg("disabling parallel option of vacuum on \"%s\" --- cannot vacuum temporary tables in parallel",
- vacrelstats->relname)));
+ vacrelstats->errinfo.relname)));
}
else
lps = begin_parallel_vacuum(RelationGetRelid(onerel), Irel,
@@ -948,7 +953,8 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_SCANNED, blkno);
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_SCAN_HEAP,
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ VACUUM_ERRCB_PHASE_SCAN_HEAP,
blkno, NULL);
if (blkno == next_unskippable_block)
@@ -1582,7 +1588,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
&& VM_ALL_VISIBLE(onerel, blkno, &vmbuffer))
{
elog(WARNING, "page is not marked all-visible but visibility map bit is set in relation \"%s\" page %u",
- vacrelstats->relname, blkno);
+ vacrelstats->errinfo.relname, blkno);
visibilitymap_clear(onerel, blkno, vmbuffer,
VISIBILITYMAP_VALID_BITS);
}
@@ -1603,7 +1609,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
else if (PageIsAllVisible(page) && has_dead_tuples)
{
elog(WARNING, "page containing dead tuples is marked as all-visible in relation \"%s\" page %u",
- vacrelstats->relname, blkno);
+ vacrelstats->errinfo.relname, blkno);
PageClearAllVisible(page);
MarkBufferDirty(buf);
visibilitymap_clear(onerel, blkno, vmbuffer,
@@ -1713,7 +1719,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
if (vacuumed_pages)
ereport(elevel,
(errmsg("\"%s\": removed %.0f row versions in %u pages",
- vacrelstats->relname,
+ vacrelstats->errinfo.relname,
tups_vacuumed, vacuumed_pages)));
/*
@@ -1742,7 +1748,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
ereport(elevel,
(errmsg("\"%s\": found %.0f removable, %.0f nonremovable row versions in %u out of %u pages",
- vacrelstats->relname,
+ vacrelstats->errinfo.relname,
tups_vacuumed, num_tuples,
vacrelstats->scanned_pages, nblocks),
errdetail_internal("%s", buf.data)));
@@ -1820,15 +1826,16 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
int npages;
PGRUsage ru0;
Buffer vmbuffer = InvalidBuffer;
- LVRelStats olderrinfo;
+ LVSavedPosition olderrinfo;
/* Report that we are now vacuuming the heap */
pgstat_progress_update_param(PROGRESS_VACUUM_PHASE,
PROGRESS_VACUUM_PHASE_VACUUM_HEAP);
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ olderrinfo = vacrelstats->errinfo;
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ VACUUM_ERRCB_PHASE_VACUUM_HEAP,
InvalidBlockNumber, NULL);
pg_rusage_init(&ru0);
@@ -1845,7 +1852,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
vacuum_delay_point();
tblk = ItemPointerGetBlockNumber(&vacrelstats->dead_tuples->itemptrs[tupindex]);
- vacrelstats->blkno = tblk;
+ vacrelstats->errinfo.blkno = tblk;
buf = ReadBufferExtended(onerel, MAIN_FORKNUM, tblk, RBM_NORMAL,
vac_strategy);
if (!ConditionalLockBufferForCleanup(buf))
@@ -1874,12 +1881,12 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
ereport(elevel,
(errmsg("\"%s\": removed %d row versions in %d pages",
- vacrelstats->relname,
+ vacrelstats->errinfo.relname,
tupindex, npages),
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
+ update_vacuum_error_info(&vacrelstats->errinfo,
olderrinfo.phase,
olderrinfo.blkno,
olderrinfo.indname);
@@ -1905,13 +1912,14 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
- LVRelStats olderrinfo;
+ LVSavedPosition olderrinfo;
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ olderrinfo = vacrelstats->errinfo;
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ VACUUM_ERRCB_PHASE_VACUUM_HEAP,
blkno, NULL);
START_CRIT_SECTION();
@@ -1991,7 +1999,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
}
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
+ update_vacuum_error_info(&vacrelstats->errinfo,
olderrinfo.phase,
olderrinfo.blkno,
olderrinfo.indname);
@@ -2404,7 +2412,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrinfo;
+ LVSavedPosition olderrinfo;
pg_rusage_init(&ru0);
@@ -2417,8 +2425,8 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats,
+ olderrinfo = vacrelstats->errinfo;
+ update_vacuum_error_info(&vacrelstats->errinfo,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber,
RelationGetRelationName(indrel));
@@ -2434,12 +2442,12 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ereport(elevel,
(errmsg(msg,
- vacrelstats->indname,
+ vacrelstats->errinfo.indname,
dead_tuples->num_tuples),
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
+ update_vacuum_error_info(&vacrelstats->errinfo,
olderrinfo.phase,
olderrinfo.blkno,
olderrinfo.indname);
@@ -2459,7 +2467,7 @@ lazy_cleanup_index(Relation indrel,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrinfo;
+ LVSavedPosition olderrinfo;
pg_rusage_init(&ru0);
@@ -2473,8 +2481,8 @@ lazy_cleanup_index(Relation indrel,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats,
+ olderrinfo = vacrelstats->errinfo;
+ update_vacuum_error_info(&vacrelstats->errinfo,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber,
RelationGetRelationName(indrel));
@@ -2482,7 +2490,7 @@ lazy_cleanup_index(Relation indrel,
*stats = index_vacuum_cleanup(&ivinfo, *stats);
/* Revert back to the old phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
+ update_vacuum_error_info(&vacrelstats->errinfo,
olderrinfo.phase,
olderrinfo.blkno,
olderrinfo.indname);
@@ -2597,7 +2605,7 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
vacrelstats->lock_waiter_detected = true;
ereport(elevel,
(errmsg("\"%s\": stopping truncate due to conflicting lock request",
- vacrelstats->relname)));
+ vacrelstats->errinfo.relname)));
return;
}
@@ -2630,7 +2638,7 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
* were vacuuming.
*/
new_rel_pages = count_nondeletable_pages(onerel, vacrelstats);
- vacrelstats->blkno = new_rel_pages;
+ vacrelstats->errinfo.blkno = new_rel_pages;
if (new_rel_pages >= old_rel_pages)
{
@@ -2663,7 +2671,7 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
ereport(elevel,
(errmsg("\"%s\": truncated %u to %u pages",
- vacrelstats->relname,
+ vacrelstats->errinfo.relname,
old_rel_pages, new_rel_pages),
errdetail_internal("%s",
pg_rusage_show(&ru0))));
@@ -2728,7 +2736,7 @@ count_nondeletable_pages(Relation onerel, LVRelStats *vacrelstats)
{
ereport(elevel,
(errmsg("\"%s\": suspending truncate due to conflicting lock request",
- vacrelstats->relname)));
+ vacrelstats->errinfo.relname)));
vacrelstats->lock_waiter_detected = true;
return blkno;
@@ -3521,14 +3529,14 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
* Initialize vacrelstats for use as error callback arg by parallel
* worker.
*/
- vacrelstats.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
- vacrelstats.relname = pstrdup(RelationGetRelationName(onerel));
- vacrelstats.indname = NULL;
- vacrelstats.phase = VACUUM_ERRCB_PHASE_UNKNOWN; /* Not yet processing */
+ vacrelstats.errinfo.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
+ vacrelstats.errinfo.relname = pstrdup(RelationGetRelationName(onerel));
+ vacrelstats.errinfo.indname = NULL;
+ vacrelstats.errinfo.phase = VACUUM_ERRCB_PHASE_UNKNOWN; /* Not yet processing */
/* Setup error traceback support for ereport() */
errcallback.callback = vacuum_error_callback;
- errcallback.arg = &vacrelstats;
+ errcallback.arg = &vacrelstats.errinfo;
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
@@ -3559,7 +3567,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
static void
vacuum_error_callback(void *arg)
{
- LVRelStats *errinfo = arg;
+ LVSavedPosition *errinfo = arg;
switch (errinfo->phase)
{
@@ -3600,7 +3608,7 @@ vacuum_error_callback(void *arg)
/* Update vacuum error callback for the current phase, block, and index. */
static void
-update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno,
+update_vacuum_error_info(LVSavedPosition *errinfo, int phase, BlockNumber blkno,
char *indname)
{
errinfo->blkno = blkno;
--
2.17.0
0004-Update-functions-to-pass-only-errinfo-struct.patchtext/x-diff; charset=us-asciiDownload
From 2b98df134ed4b84ed22cb7b749dc43fa43e2f370 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 22 Jun 2020 17:13:39 -0500
Subject: [PATCH 4/4] Update functions to pass only errinfo struct..
..this is a more complete change, and probably a good idea since parallel
workers have an partially-populated LVRelStats, and it's better to avoid the
idea that it's usable (dead_tuples in particular is a bogus pointer).
---
src/backend/access/heap/vacuumlazy.c | 58 ++++++++++++++--------------
1 file changed, 29 insertions(+), 29 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 6bd2867660..db3bd86338 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -344,10 +344,10 @@ static void lazy_vacuum_all_indexes(Relation onerel, Relation *Irel,
LVRelStats *vacrelstats, LVParallelState *lps,
int nindexes);
static void lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
- LVDeadTuples *dead_tuples, double reltuples, LVRelStats *vacrelstats);
+ LVDeadTuples *dead_tuples, double reltuples, LVSavedPosition *errinfo);
static void lazy_cleanup_index(Relation indrel,
IndexBulkDeleteResult **stats,
- double reltuples, bool estimated_count, LVRelStats *vacrelstats);
+ double reltuples, bool estimated_count, LVSavedPosition *errinfo);
static int lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int tupindex, LVRelStats *vacrelstats, Buffer *vmbuffer);
static bool should_attempt_truncation(VacuumParams *params,
@@ -367,13 +367,13 @@ static void lazy_parallel_vacuum_indexes(Relation *Irel, IndexBulkDeleteResult *
int nindexes);
static void parallel_vacuum_index(Relation *Irel, IndexBulkDeleteResult **stats,
LVShared *lvshared, LVDeadTuples *dead_tuples,
- int nindexes, LVRelStats *vacrelstats);
+ int nindexes, LVSavedPosition *errinfo);
static void vacuum_indexes_leader(Relation *Irel, IndexBulkDeleteResult **stats,
LVRelStats *vacrelstats, LVParallelState *lps,
int nindexes);
static void vacuum_one_index(Relation indrel, IndexBulkDeleteResult **stats,
LVShared *lvshared, LVSharedIndStats *shared_indstats,
- LVDeadTuples *dead_tuples, LVRelStats *vacrelstats);
+ LVDeadTuples *dead_tuples, LVSavedPosition *errinfo);
static void lazy_cleanup_all_indexes(Relation *Irel, IndexBulkDeleteResult **stats,
LVRelStats *vacrelstats, LVParallelState *lps,
int nindexes);
@@ -1798,7 +1798,7 @@ lazy_vacuum_all_indexes(Relation onerel, Relation *Irel,
for (idx = 0; idx < nindexes; idx++)
lazy_vacuum_index(Irel[idx], &stats[idx], vacrelstats->dead_tuples,
- vacrelstats->old_live_tuples, vacrelstats);
+ vacrelstats->old_live_tuples, &vacrelstats->errinfo);
}
/* Increase and report the number of index scans */
@@ -2163,7 +2163,7 @@ lazy_parallel_vacuum_indexes(Relation *Irel, IndexBulkDeleteResult **stats,
* indexes in the case where no workers are launched.
*/
parallel_vacuum_index(Irel, stats, lps->lvshared,
- vacrelstats->dead_tuples, nindexes, vacrelstats);
+ vacrelstats->dead_tuples, nindexes, &vacrelstats->errinfo);
/*
* Next, accumulate buffer and WAL usage. (This must wait for the workers
@@ -2198,7 +2198,7 @@ lazy_parallel_vacuum_indexes(Relation *Irel, IndexBulkDeleteResult **stats,
static void
parallel_vacuum_index(Relation *Irel, IndexBulkDeleteResult **stats,
LVShared *lvshared, LVDeadTuples *dead_tuples,
- int nindexes, LVRelStats *vacrelstats)
+ int nindexes, LVSavedPosition *errinfo)
{
/*
* Increment the active worker count if we are able to launch any worker.
@@ -2232,7 +2232,7 @@ parallel_vacuum_index(Relation *Irel, IndexBulkDeleteResult **stats,
/* Do vacuum or cleanup of the index */
vacuum_one_index(Irel[idx], &(stats[idx]), lvshared, shared_indstats,
- dead_tuples, vacrelstats);
+ dead_tuples, errinfo);
}
/*
@@ -2273,7 +2273,7 @@ vacuum_indexes_leader(Relation *Irel, IndexBulkDeleteResult **stats,
skip_parallel_vacuum_index(Irel[i], lps->lvshared))
vacuum_one_index(Irel[i], &(stats[i]), lps->lvshared,
shared_indstats, vacrelstats->dead_tuples,
- vacrelstats);
+ &vacrelstats->errinfo);
}
/*
@@ -2293,7 +2293,7 @@ vacuum_indexes_leader(Relation *Irel, IndexBulkDeleteResult **stats,
static void
vacuum_one_index(Relation indrel, IndexBulkDeleteResult **stats,
LVShared *lvshared, LVSharedIndStats *shared_indstats,
- LVDeadTuples *dead_tuples, LVRelStats *vacrelstats)
+ LVDeadTuples *dead_tuples, LVSavedPosition *errinfo)
{
IndexBulkDeleteResult *bulkdelete_res = NULL;
@@ -2313,10 +2313,10 @@ vacuum_one_index(Relation indrel, IndexBulkDeleteResult **stats,
/* Do vacuum or cleanup of the index */
if (lvshared->for_cleanup)
lazy_cleanup_index(indrel, stats, lvshared->reltuples,
- lvshared->estimated_count, vacrelstats);
+ lvshared->estimated_count, errinfo);
else
lazy_vacuum_index(indrel, stats, dead_tuples,
- lvshared->reltuples, vacrelstats);
+ lvshared->reltuples, errinfo);
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
@@ -2392,7 +2392,7 @@ lazy_cleanup_all_indexes(Relation *Irel, IndexBulkDeleteResult **stats,
lazy_cleanup_index(Irel[idx], &stats[idx],
vacrelstats->new_rel_tuples,
vacrelstats->tupcount_pages < vacrelstats->rel_pages,
- vacrelstats);
+ &vacrelstats->errinfo);
}
}
@@ -2407,7 +2407,7 @@ lazy_cleanup_all_indexes(Relation *Irel, IndexBulkDeleteResult **stats,
*/
static void
lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
- LVDeadTuples *dead_tuples, double reltuples, LVRelStats *vacrelstats)
+ LVDeadTuples *dead_tuples, double reltuples, LVSavedPosition *errinfo)
{
IndexVacuumInfo ivinfo;
const char *msg;
@@ -2425,8 +2425,8 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrinfo = vacrelstats->errinfo;
- update_vacuum_error_info(&vacrelstats->errinfo,
+ olderrinfo = *errinfo;
+ update_vacuum_error_info(errinfo,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber,
RelationGetRelationName(indrel));
@@ -2442,12 +2442,12 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ereport(elevel,
(errmsg(msg,
- vacrelstats->errinfo.indname,
+ errinfo->indname,
dead_tuples->num_tuples),
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(&vacrelstats->errinfo,
+ update_vacuum_error_info(errinfo,
olderrinfo.phase,
olderrinfo.blkno,
olderrinfo.indname);
@@ -2462,7 +2462,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
static void
lazy_cleanup_index(Relation indrel,
IndexBulkDeleteResult **stats,
- double reltuples, bool estimated_count, LVRelStats *vacrelstats)
+ double reltuples, bool estimated_count, LVSavedPosition *errinfo)
{
IndexVacuumInfo ivinfo;
const char *msg;
@@ -2481,8 +2481,8 @@ lazy_cleanup_index(Relation indrel,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrinfo = vacrelstats->errinfo;
- update_vacuum_error_info(&vacrelstats->errinfo,
+ olderrinfo = *errinfo;
+ update_vacuum_error_info(errinfo,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber,
RelationGetRelationName(indrel));
@@ -2490,7 +2490,7 @@ lazy_cleanup_index(Relation indrel,
*stats = index_vacuum_cleanup(&ivinfo, *stats);
/* Revert back to the old phase information for error traceback */
- update_vacuum_error_info(&vacrelstats->errinfo,
+ update_vacuum_error_info(errinfo,
olderrinfo.phase,
olderrinfo.blkno,
olderrinfo.indname);
@@ -3474,7 +3474,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
int nindexes;
char *sharedquery;
IndexBulkDeleteResult **stats;
- LVRelStats vacrelstats;
+ LVSavedPosition errinfo;
ErrorContextCallback errcallback;
lvshared = (LVShared *) shm_toc_lookup(toc, PARALLEL_VACUUM_KEY_SHARED,
@@ -3529,14 +3529,14 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
* Initialize vacrelstats for use as error callback arg by parallel
* worker.
*/
- vacrelstats.errinfo.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
- vacrelstats.errinfo.relname = pstrdup(RelationGetRelationName(onerel));
- vacrelstats.errinfo.indname = NULL;
- vacrelstats.errinfo.phase = VACUUM_ERRCB_PHASE_UNKNOWN; /* Not yet processing */
+ errinfo.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
+ errinfo.relname = pstrdup(RelationGetRelationName(onerel));
+ errinfo.indname = NULL;
+ errinfo.phase = VACUUM_ERRCB_PHASE_UNKNOWN; /* Not yet processing */
/* Setup error traceback support for ereport() */
errcallback.callback = vacuum_error_callback;
- errcallback.arg = &vacrelstats.errinfo;
+ errcallback.arg = &errinfo;
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
@@ -3545,7 +3545,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Process indexes to perform vacuum/cleanup */
parallel_vacuum_index(indrels, stats, lvshared, dead_tuples, nindexes,
- &vacrelstats);
+ &errinfo);
/* Report buffer/WAL usage during parallel execution */
buffer_usage = shm_toc_lookup(toc, PARALLEL_VACUUM_KEY_BUFFER_USAGE, false);
--
2.17.0
On Tue, Jun 23, 2020 at 7:13 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Jun 22, 2020 at 01:57:12PM -0700, Andres Freund wrote:
No, I don't think that's a solution. I think it's wrong to have
something like olderrinfo in the first place. Using a struct with ~25
members to store the current state of three variables just doesn't make
sense. Why isn't this just a LVSavedPosition struct or something like
that?I'd used LVRelStats on your suggestion:
/messages/by-id/20191211165425.4ewww2s5k5cafi4l@alap3.anarazel.de
/messages/by-id/20200120191305.sxi44cedhtxwr3ag@alap3.anarazel.deI understood the goal to be avoiding the need to add a new struct, when most
functions are already passed LVRelStats *vacrelstats.
Yeah, I think this is a good point against adding a separate struct.
I also don't think that we can buy much by doing this optimization.
To me, the current code looks good in this regard.
But maybe I misunderstood. (Also, back in January, the callback was only used
for scan-heap phase, so it's increased in scope several times).Anyway, I put together some patches for discussion purposes.
Few comments for 0002-Add-assert-and-document-why-indname-is-safe
-----------------------------------------------------------------------------------------------------
- /* Free index name from any previous phase */
if (errinfo->indname)
+ {
+ /*
+ * indname is only ever saved during lazy_vacuum_index(), which
+ * during which the phase information is not not further
+ * manipulated, until it's restored before returning from
+ * lazy_vacuum_index().
+ */
+ Assert(indname == NULL);
+
pfree(errinfo->indname);
+ errinfo->indname = NULL;
+ }
It is not very clear that this is the place where we are saving the
state. I think it would be better to do in the caller (ex. in before
statement olderrinfo = *vacrelstats; in lazy_vacuum_index()) where it
is clear that we are saving the state for later use.
I guess for the restore case we are already assigning NULL via
"errinfo->indname = indname ? pstrdup(indname) : NULL;" in
update_vacuum_error_info. I think some more comments in the function
update_vacuum_error_info would explain it better.
0001-Rename-from-errcbarg, looks fine to me but we can see if others
have any opinion on the naming (especially changing VACUUM_ERRCB* to
VACUUM_ERRINFO*).
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
Hi,
On 2020-06-22 20:43:47 -0500, Justin Pryzby wrote:
On Mon, Jun 22, 2020 at 01:57:12PM -0700, Andres Freund wrote:
On 2020-06-22 15:43:11 -0500, Justin Pryzby wrote:
On Mon, Jun 22, 2020 at 01:09:39PM -0700, Andres Freund wrote:
I'm also uncomfortable with the approach of just copying all of
LVRelStats in several places:/*
@@ -1580,9 +1648,15 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
+ LVRelStats olderrinfo;I guess the alternative is to write like
LVRelStats olderrinfo = {
.phase = vacrelstats.phase,
.blkno = vacrelstats.blkno,
.indname = vacrelstats.indname,
};No, I don't think that's a solution. I think it's wrong to have
something like olderrinfo in the first place. Using a struct with ~25
members to store the current state of three variables just doesn't make
sense. Why isn't this just a LVSavedPosition struct or something like
that?I'd used LVRelStats on your suggestion:
/messages/by-id/20191211165425.4ewww2s5k5cafi4l@alap3.anarazel.de
/messages/by-id/20200120191305.sxi44cedhtxwr3ag@alap3.anarazel.deI understood the goal to be avoiding the need to add a new struct, when most
functions are already passed LVRelStats *vacrelstats.
But maybe I misunderstood. (Also, back in January, the callback was only used
for scan-heap phase, so it's increased in scope several times).
I am only suggesting that where you save the old location, as currently
done with LVRelStats olderrinfo, you instead use a more specific
type. Not that you should pass that anywhere (except for
update_vacuum_error_info).
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
I am only suggesting that where you save the old location, as currently
done with LVRelStats olderrinfo, you instead use a more specific
type. Not that you should pass that anywhere (except for
update_vacuum_error_info).
As things currently stand, I don't think we need another struct
type at all. ISTM we should hard-wire the handling of indname
as I suggested above. Then there are only two fields to be dealt
with, and we could just as well save them in simple local variables.
If there's a clear future path to needing to save/restore more
fields, then maybe another struct type would be useful ... but
right now the struct type declaration itself would take more
lines of code than it would save.
regards, tom lane
On Tue, Jun 23, 2020 at 12:14:40AM -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
I am only suggesting that where you save the old location, as currently
done with LVRelStats olderrinfo, you instead use a more specific
type. Not that you should pass that anywhere (except for
update_vacuum_error_info).As things currently stand, I don't think we need another struct
type at all. ISTM we should hard-wire the handling of indname
as I suggested above. Then there are only two fields to be dealt
with, and we could just as well save them in simple local variables.If there's a clear future path to needing to save/restore more
fields, then maybe another struct type would be useful ... but
right now the struct type declaration itself would take more
lines of code than it would save.
Updated patches for consideration. I left the "struct" patch there to show
what it'd look like.
--
Justin
Attachments:
v2-0001-Rename-from-errcbarg.patchtext/x-diff; charset=us-asciiDownload
From bbfdff2483730d45c663b75de4700e50f09ab4d3 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 22 Jun 2020 18:22:58 -0500
Subject: [PATCH v2 1/5] Rename from "errcbarg"..
..the name of which Alvaro didn't like. I renamed this, but it was
accidentally re-introduced while trading patches with Amit.
We should *also* rename VACUUM_ERRCB* to VACUUM_ERRINFO, but I didn't do that here.
---
src/backend/access/heap/vacuumlazy.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3bef0e124b..a6a5d906c0 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -2459,7 +2459,7 @@ lazy_cleanup_index(Relation indrel,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrcbarg;
+ LVRelStats olderrinfo;
pg_rusage_init(&ru0);
@@ -2473,7 +2473,7 @@ lazy_cleanup_index(Relation indrel,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrcbarg = *vacrelstats;
+ olderrinfo = *vacrelstats;
update_vacuum_error_info(vacrelstats,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber,
@@ -2483,9 +2483,9 @@ lazy_cleanup_index(Relation indrel,
/* Revert back to the old phase information for error traceback */
update_vacuum_error_info(vacrelstats,
- olderrcbarg.phase,
- olderrcbarg.blkno,
- olderrcbarg.indname);
+ olderrinfo.phase,
+ olderrinfo.blkno,
+ olderrinfo.indname);
if (!(*stats))
return;
--
2.17.0
v2-0002-Specially-save-the-index-name-in-lazy_-vacuum-cle.patchtext/x-diff; charset=us-asciiDownload
From afd4916d190d7da48b535432f8edea65615d7fe8 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 22 Jun 2020 18:51:35 -0500
Subject: [PATCH v2 2/5] Specially save the index name in
lazy_{vacuum,cleanup}_index..
..at expense of symmetry, move handling of index outside of
update_vacuum_error_info, since it's more clear this way that use of indname is
limited and safe.
---
src/backend/access/heap/vacuumlazy.c | 49 ++++++++++++----------------
1 file changed, 20 insertions(+), 29 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index a6a5d906c0..43a3c093fd 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -389,7 +389,7 @@ static LVSharedIndStats *get_indstats(LVShared *lvshared, int n);
static bool skip_parallel_vacuum_index(Relation indrel, LVShared *lvshared);
static void vacuum_error_callback(void *arg);
static void update_vacuum_error_info(LVRelStats *errinfo, int phase,
- BlockNumber blkno, char *indname);
+ BlockNumber blkno);
/*
@@ -477,7 +477,6 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
vacrelstats->relnamespace = get_namespace_name(RelationGetNamespace(onerel));
vacrelstats->relname = pstrdup(RelationGetRelationName(onerel));
- vacrelstats->indname = NULL;
vacrelstats->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrelstats->old_rel_pages = onerel->rd_rel->relpages;
vacrelstats->old_live_tuples = onerel->rd_rel->reltuples;
@@ -539,7 +538,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
* revert to the previous phase.
*/
update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_TRUNCATE,
- vacrelstats->nonempty_pages, NULL);
+ vacrelstats->nonempty_pages);
lazy_truncate_heap(onerel, vacrelstats);
}
@@ -949,7 +948,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_SCANNED, blkno);
update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_SCAN_HEAP,
- blkno, NULL);
+ blkno);
if (blkno == next_unskippable_block)
{
@@ -1829,7 +1828,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
/* Update error traceback information */
olderrinfo = *vacrelstats;
update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
- InvalidBlockNumber, NULL);
+ InvalidBlockNumber);
pg_rusage_init(&ru0);
npages = 0;
@@ -1881,8 +1880,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
/* Revert to the previous phase information for error traceback */
update_vacuum_error_info(vacrelstats,
olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ olderrinfo.blkno);
}
/*
@@ -1912,7 +1910,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
/* Update error traceback information */
olderrinfo = *vacrelstats;
update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
- blkno, NULL);
+ blkno);
START_CRIT_SECTION();
@@ -1993,8 +1991,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
/* Revert to the previous phase information for error traceback */
update_vacuum_error_info(vacrelstats,
olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ olderrinfo.blkno);
return tupindex;
}
@@ -2418,10 +2415,11 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
/* Update error traceback information */
olderrinfo = *vacrelstats;
+ /* The index name is also saved during this phase */
+ vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
update_vacuum_error_info(vacrelstats,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
- InvalidBlockNumber,
- RelationGetRelationName(indrel));
+ InvalidBlockNumber);
/* Do bulk deletion */
*stats = index_bulk_delete(&ivinfo, *stats,
@@ -2441,8 +2439,8 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
/* Revert to the previous phase information for error traceback */
update_vacuum_error_info(vacrelstats,
olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ olderrinfo.blkno);
+ pfree(vacrelstats->indname);
}
/*
@@ -2474,18 +2472,20 @@ lazy_cleanup_index(Relation indrel,
/* Update error traceback information */
olderrinfo = *vacrelstats;
+ /* The index name is also saved during this phase */
+ vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
update_vacuum_error_info(vacrelstats,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
- InvalidBlockNumber,
- RelationGetRelationName(indrel));
+ InvalidBlockNumber);
*stats = index_vacuum_cleanup(&ivinfo, *stats);
/* Revert back to the old phase information for error traceback */
update_vacuum_error_info(vacrelstats,
olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ olderrinfo.blkno);
+ pfree(vacrelstats->indname);
+
if (!(*stats))
return;
@@ -3523,7 +3523,6 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
*/
vacrelstats.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
vacrelstats.relname = pstrdup(RelationGetRelationName(onerel));
- vacrelstats.indname = NULL;
vacrelstats.phase = VACUUM_ERRCB_PHASE_UNKNOWN; /* Not yet processing */
/* Setup error traceback support for ereport() */
@@ -3598,18 +3597,10 @@ vacuum_error_callback(void *arg)
}
}
-/* Update vacuum error callback for the current phase, block, and index. */
+/* Update vacuum error callback for the current phase and block. */
static void
-update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno,
- char *indname)
+update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno)
{
errinfo->blkno = blkno;
errinfo->phase = phase;
-
- /* Free index name from any previous phase */
- if (errinfo->indname)
- pfree(errinfo->indname);
-
- /* For index phases, save the name of the current index for the callback */
- errinfo->indname = indname ? pstrdup(indname) : NULL;
}
--
2.17.0
v2-0003-Save-phase-blkno-in-local-variables-for-consisten.patchtext/x-diff; charset=us-asciiDownload
From f57302143cea857b29b4f19f30bad0e28209c50d Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Tue, 23 Jun 2020 08:35:29 -0500
Subject: [PATCH v2 3/5] Save phase/blkno in local variables, for consistency
with indname
---
src/backend/access/heap/vacuumlazy.c | 32 +++++++++++-----------------
1 file changed, 12 insertions(+), 20 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 43a3c093fd..2393269565 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1819,14 +1819,14 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
int npages;
PGRUsage ru0;
Buffer vmbuffer = InvalidBuffer;
- LVRelStats olderrinfo;
+ int oldphase = vacrelstats->phase,
+ oldblkno = vacrelstats->blkno;
/* Report that we are now vacuuming the heap */
pgstat_progress_update_param(PROGRESS_VACUUM_PHASE,
PROGRESS_VACUUM_PHASE_VACUUM_HEAP);
/* Update error traceback information */
- olderrinfo = *vacrelstats;
update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
InvalidBlockNumber);
@@ -1878,9 +1878,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno);
+ update_vacuum_error_info(vacrelstats, oldphase, oldblkno);
}
/*
@@ -1903,12 +1901,12 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
- LVRelStats olderrinfo;
+ int oldphase = vacrelstats->phase,
+ oldblkno = vacrelstats->blkno;
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
/* Update error traceback information */
- olderrinfo = *vacrelstats;
update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
blkno);
@@ -1989,9 +1987,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
}
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno);
+ update_vacuum_error_info(vacrelstats, oldphase, oldblkno);
return tupindex;
}
@@ -2401,7 +2397,8 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrinfo;
+ int oldphase = vacrelstats->phase,
+ oldblkno = vacrelstats->blkno;
pg_rusage_init(&ru0);
@@ -2414,7 +2411,6 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrinfo = *vacrelstats;
/* The index name is also saved during this phase */
vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
update_vacuum_error_info(vacrelstats,
@@ -2437,9 +2433,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno);
+ update_vacuum_error_info(vacrelstats, oldphase, oldblkno);
pfree(vacrelstats->indname);
}
@@ -2457,7 +2451,8 @@ lazy_cleanup_index(Relation indrel,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrinfo;
+ int oldphase = vacrelstats->phase,
+ oldblkno = vacrelstats->blkno;
pg_rusage_init(&ru0);
@@ -2471,7 +2466,6 @@ lazy_cleanup_index(Relation indrel,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrinfo = *vacrelstats;
/* The index name is also saved during this phase */
vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
update_vacuum_error_info(vacrelstats,
@@ -2481,9 +2475,7 @@ lazy_cleanup_index(Relation indrel,
*stats = index_vacuum_cleanup(&ivinfo, *stats);
/* Revert back to the old phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno);
+ update_vacuum_error_info(vacrelstats, oldphase, oldblkno);
pfree(vacrelstats->indname);
if (!(*stats))
--
2.17.0
v2-0004-Move-error-information-into-a-separate-struct.patchtext/x-diff; charset=us-asciiDownload
From 78d378fd22f74828981692e5e253214bccef4311 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 22 Jun 2020 16:36:14 -0500
Subject: [PATCH v2 4/5] Move error information into a separate struct..
..to avoid copying large LVRelStats struct.
Andres things this is a problem but Tom thinks it's a micro-optimization.
See: b61d161c146328ae6ba9ed937862d66e5c8b035a
This also moves relname and relnamespace (see ef75140fe).
Discussion: https://www.postgresql.org/message-id/20200622200939.jkuniq3vtiumeszj%40alap3.anarazel.de
---
src/backend/access/heap/vacuumlazy.c | 128 +++++++++++++++------------
1 file changed, 72 insertions(+), 56 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 2393269565..9eb4bc66ae 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -288,10 +288,17 @@ typedef struct LVParallelState
int nindexes_parallel_condcleanup;
} LVParallelState;
-typedef struct LVRelStats
+typedef struct
{
char *relnamespace;
char *relname;
+ char *indname;
+ BlockNumber blkno; /* used only for heap operations */
+ VacErrPhase phase;
+} LVSavedPosition;
+
+typedef struct LVRelStats
+{
/* useindex = true means two-pass strategy; false means one-pass */
bool useindex;
/* Overall statistics about rel */
@@ -313,10 +320,7 @@ typedef struct LVRelStats
TransactionId latestRemovedXid;
bool lock_waiter_detected;
- /* Used for error callback */
- char *indname;
- BlockNumber blkno; /* used only for heap operations */
- VacErrPhase phase;
+ LVSavedPosition errinfo; /* Used for error callback */
} LVRelStats;
/* A few variables that don't seem worth passing around as parameters */
@@ -388,7 +392,7 @@ static void end_parallel_vacuum(Relation *Irel, IndexBulkDeleteResult **stats,
static LVSharedIndStats *get_indstats(LVShared *lvshared, int n);
static bool skip_parallel_vacuum_index(Relation indrel, LVShared *lvshared);
static void vacuum_error_callback(void *arg);
-static void update_vacuum_error_info(LVRelStats *errinfo, int phase,
+static void update_vacuum_error_info(LVSavedPosition *errinfo, int phase,
BlockNumber blkno);
@@ -475,9 +479,9 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
vacrelstats = (LVRelStats *) palloc0(sizeof(LVRelStats));
- vacrelstats->relnamespace = get_namespace_name(RelationGetNamespace(onerel));
- vacrelstats->relname = pstrdup(RelationGetRelationName(onerel));
- vacrelstats->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
+ vacrelstats->errinfo.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
+ vacrelstats->errinfo.relname = pstrdup(RelationGetRelationName(onerel));
+ vacrelstats->errinfo.phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrelstats->old_rel_pages = onerel->rd_rel->relpages;
vacrelstats->old_live_tuples = onerel->rd_rel->reltuples;
vacrelstats->num_index_scans = 0;
@@ -501,7 +505,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
* information is restored at the end of those phases.
*/
errcallback.callback = vacuum_error_callback;
- errcallback.arg = vacrelstats;
+ errcallback.arg = &vacrelstats->errinfo;
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
@@ -537,7 +541,8 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
* which we add context information to errors, so we don't need to
* revert to the previous phase.
*/
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_TRUNCATE,
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ VACUUM_ERRCB_PHASE_TRUNCATE,
vacrelstats->nonempty_pages);
lazy_truncate_heap(onerel, vacrelstats);
}
@@ -649,8 +654,8 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
}
appendStringInfo(&buf, msgfmt,
get_database_name(MyDatabaseId),
- vacrelstats->relnamespace,
- vacrelstats->relname,
+ vacrelstats->errinfo.relnamespace,
+ vacrelstats->errinfo.relname,
vacrelstats->num_index_scans);
appendStringInfo(&buf, _("pages: %u removed, %u remain, %u skipped due to pins, %u skipped frozen\n"),
vacrelstats->pages_removed,
@@ -785,13 +790,13 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
if (aggressive)
ereport(elevel,
(errmsg("aggressively vacuuming \"%s.%s\"",
- vacrelstats->relnamespace,
- vacrelstats->relname)));
+ vacrelstats->errinfo.relnamespace,
+ vacrelstats->errinfo.relname)));
else
ereport(elevel,
(errmsg("vacuuming \"%s.%s\"",
- vacrelstats->relnamespace,
- vacrelstats->relname)));
+ vacrelstats->errinfo.relnamespace,
+ vacrelstats->errinfo.relname)));
empty_pages = vacuumed_pages = 0;
next_fsm_block_to_vacuum = (BlockNumber) 0;
@@ -827,7 +832,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
if (params->nworkers > 0)
ereport(WARNING,
(errmsg("disabling parallel option of vacuum on \"%s\" --- cannot vacuum temporary tables in parallel",
- vacrelstats->relname)));
+ vacrelstats->errinfo.relname)));
}
else
lps = begin_parallel_vacuum(RelationGetRelid(onerel), Irel,
@@ -947,7 +952,8 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_SCANNED, blkno);
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_SCAN_HEAP,
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ VACUUM_ERRCB_PHASE_SCAN_HEAP,
blkno);
if (blkno == next_unskippable_block)
@@ -1581,7 +1587,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
&& VM_ALL_VISIBLE(onerel, blkno, &vmbuffer))
{
elog(WARNING, "page is not marked all-visible but visibility map bit is set in relation \"%s\" page %u",
- vacrelstats->relname, blkno);
+ vacrelstats->errinfo.relname, blkno);
visibilitymap_clear(onerel, blkno, vmbuffer,
VISIBILITYMAP_VALID_BITS);
}
@@ -1602,7 +1608,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
else if (PageIsAllVisible(page) && has_dead_tuples)
{
elog(WARNING, "page containing dead tuples is marked as all-visible in relation \"%s\" page %u",
- vacrelstats->relname, blkno);
+ vacrelstats->errinfo.relname, blkno);
PageClearAllVisible(page);
MarkBufferDirty(buf);
visibilitymap_clear(onerel, blkno, vmbuffer,
@@ -1712,7 +1718,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
if (vacuumed_pages)
ereport(elevel,
(errmsg("\"%s\": removed %.0f row versions in %u pages",
- vacrelstats->relname,
+ vacrelstats->errinfo.relname,
tups_vacuumed, vacuumed_pages)));
/*
@@ -1741,7 +1747,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
ereport(elevel,
(errmsg("\"%s\": found %.0f removable, %.0f nonremovable row versions in %u out of %u pages",
- vacrelstats->relname,
+ vacrelstats->errinfo.relname,
tups_vacuumed, num_tuples,
vacrelstats->scanned_pages, nblocks),
errdetail_internal("%s", buf.data)));
@@ -1819,15 +1825,16 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
int npages;
PGRUsage ru0;
Buffer vmbuffer = InvalidBuffer;
- int oldphase = vacrelstats->phase,
- oldblkno = vacrelstats->blkno;
+ LVSavedPosition olderrinfo;
/* Report that we are now vacuuming the heap */
pgstat_progress_update_param(PROGRESS_VACUUM_PHASE,
PROGRESS_VACUUM_PHASE_VACUUM_HEAP);
/* Update error traceback information */
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ olderrinfo = vacrelstats->errinfo;
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ VACUUM_ERRCB_PHASE_VACUUM_HEAP,
InvalidBlockNumber);
pg_rusage_init(&ru0);
@@ -1844,7 +1851,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
vacuum_delay_point();
tblk = ItemPointerGetBlockNumber(&vacrelstats->dead_tuples->itemptrs[tupindex]);
- vacrelstats->blkno = tblk;
+ vacrelstats->errinfo.blkno = tblk;
buf = ReadBufferExtended(onerel, MAIN_FORKNUM, tblk, RBM_NORMAL,
vac_strategy);
if (!ConditionalLockBufferForCleanup(buf))
@@ -1873,12 +1880,14 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
ereport(elevel,
(errmsg("\"%s\": removed %d row versions in %d pages",
- vacrelstats->relname,
+ vacrelstats->errinfo.relname,
tupindex, npages),
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats, oldphase, oldblkno);
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ olderrinfo.phase,
+ olderrinfo.blkno);
}
/*
@@ -1901,13 +1910,14 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
- int oldphase = vacrelstats->phase,
- oldblkno = vacrelstats->blkno;
+ LVSavedPosition olderrinfo;
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
/* Update error traceback information */
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ olderrinfo = vacrelstats->errinfo;
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ VACUUM_ERRCB_PHASE_VACUUM_HEAP,
blkno);
START_CRIT_SECTION();
@@ -1987,7 +1997,9 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
}
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats, oldphase, oldblkno);
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ olderrinfo.phase,
+ olderrinfo.blkno);
return tupindex;
}
@@ -2397,8 +2409,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- int oldphase = vacrelstats->phase,
- oldblkno = vacrelstats->blkno;
+ LVSavedPosition olderrinfo;
pg_rusage_init(&ru0);
@@ -2411,9 +2422,10 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
+ olderrinfo = vacrelstats->errinfo;
/* The index name is also saved during this phase */
- vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
- update_vacuum_error_info(vacrelstats,
+ vacrelstats->errinfo.indname = pstrdup(RelationGetRelationName(indrel));
+ update_vacuum_error_info(&vacrelstats->errinfo,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber);
@@ -2428,13 +2440,15 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ereport(elevel,
(errmsg(msg,
- vacrelstats->indname,
+ vacrelstats->errinfo.indname,
dead_tuples->num_tuples),
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats, oldphase, oldblkno);
- pfree(vacrelstats->indname);
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ olderrinfo.phase,
+ olderrinfo.blkno);
+ pfree(vacrelstats->errinfo.indname);
}
/*
@@ -2451,8 +2465,7 @@ lazy_cleanup_index(Relation indrel,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- int oldphase = vacrelstats->phase,
- oldblkno = vacrelstats->blkno;
+ LVSavedPosition olderrinfo;
pg_rusage_init(&ru0);
@@ -2466,17 +2479,20 @@ lazy_cleanup_index(Relation indrel,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
+ olderrinfo = vacrelstats->errinfo;
/* The index name is also saved during this phase */
- vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
- update_vacuum_error_info(vacrelstats,
+ vacrelstats->errinfo.indname = pstrdup(RelationGetRelationName(indrel));
+ update_vacuum_error_info(&vacrelstats->errinfo,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber);
*stats = index_vacuum_cleanup(&ivinfo, *stats);
/* Revert back to the old phase information for error traceback */
- update_vacuum_error_info(vacrelstats, oldphase, oldblkno);
- pfree(vacrelstats->indname);
+ update_vacuum_error_info(&vacrelstats->errinfo,
+ olderrinfo.phase,
+ olderrinfo.blkno);
+ pfree(vacrelstats->errinfo.indname);
if (!(*stats))
return;
@@ -2589,7 +2605,7 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
vacrelstats->lock_waiter_detected = true;
ereport(elevel,
(errmsg("\"%s\": stopping truncate due to conflicting lock request",
- vacrelstats->relname)));
+ vacrelstats->errinfo.relname)));
return;
}
@@ -2622,7 +2638,7 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
* were vacuuming.
*/
new_rel_pages = count_nondeletable_pages(onerel, vacrelstats);
- vacrelstats->blkno = new_rel_pages;
+ vacrelstats->errinfo.blkno = new_rel_pages;
if (new_rel_pages >= old_rel_pages)
{
@@ -2655,7 +2671,7 @@ lazy_truncate_heap(Relation onerel, LVRelStats *vacrelstats)
ereport(elevel,
(errmsg("\"%s\": truncated %u to %u pages",
- vacrelstats->relname,
+ vacrelstats->errinfo.relname,
old_rel_pages, new_rel_pages),
errdetail_internal("%s",
pg_rusage_show(&ru0))));
@@ -2720,7 +2736,7 @@ count_nondeletable_pages(Relation onerel, LVRelStats *vacrelstats)
{
ereport(elevel,
(errmsg("\"%s\": suspending truncate due to conflicting lock request",
- vacrelstats->relname)));
+ vacrelstats->errinfo.relname)));
vacrelstats->lock_waiter_detected = true;
return blkno;
@@ -3513,13 +3529,13 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
* Initialize vacrelstats for use as error callback arg by parallel
* worker.
*/
- vacrelstats.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
- vacrelstats.relname = pstrdup(RelationGetRelationName(onerel));
- vacrelstats.phase = VACUUM_ERRCB_PHASE_UNKNOWN; /* Not yet processing */
+ vacrelstats.errinfo.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
+ vacrelstats.errinfo.relname = pstrdup(RelationGetRelationName(onerel));
+ vacrelstats.errinfo.phase = VACUUM_ERRCB_PHASE_UNKNOWN; /* Not yet processing */
/* Setup error traceback support for ereport() */
errcallback.callback = vacuum_error_callback;
- errcallback.arg = &vacrelstats;
+ errcallback.arg = &vacrelstats.errinfo;
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
@@ -3550,7 +3566,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
static void
vacuum_error_callback(void *arg)
{
- LVRelStats *errinfo = arg;
+ LVSavedPosition *errinfo = arg;
switch (errinfo->phase)
{
@@ -3591,7 +3607,7 @@ vacuum_error_callback(void *arg)
/* Update vacuum error callback for the current phase and block. */
static void
-update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno)
+update_vacuum_error_info(LVSavedPosition *errinfo, int phase, BlockNumber blkno)
{
errinfo->blkno = blkno;
errinfo->phase = phase;
--
2.17.0
v2-0005-Update-functions-to-pass-only-errinfo-struct.patchtext/x-diff; charset=us-asciiDownload
From 5e15b14c39fcbf9fce1599d0583f1aa568986e43 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 22 Jun 2020 17:13:39 -0500
Subject: [PATCH v2 5/5] Update functions to pass only errinfo struct..
..this is a more complete change, and probably a good idea since parallel
workers have an partially-populated LVRelStats, and it's better to avoid the
idea that it's usable (dead_tuples in particular is a bogus pointer).
---
src/backend/access/heap/vacuumlazy.c | 64 ++++++++++++++--------------
1 file changed, 32 insertions(+), 32 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 9eb4bc66ae..d239ad4d62 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -344,10 +344,10 @@ static void lazy_vacuum_all_indexes(Relation onerel, Relation *Irel,
LVRelStats *vacrelstats, LVParallelState *lps,
int nindexes);
static void lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
- LVDeadTuples *dead_tuples, double reltuples, LVRelStats *vacrelstats);
+ LVDeadTuples *dead_tuples, double reltuples, LVSavedPosition *errinfo);
static void lazy_cleanup_index(Relation indrel,
IndexBulkDeleteResult **stats,
- double reltuples, bool estimated_count, LVRelStats *vacrelstats);
+ double reltuples, bool estimated_count, LVSavedPosition *errinfo);
static int lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int tupindex, LVRelStats *vacrelstats, Buffer *vmbuffer);
static bool should_attempt_truncation(VacuumParams *params,
@@ -367,13 +367,13 @@ static void lazy_parallel_vacuum_indexes(Relation *Irel, IndexBulkDeleteResult *
int nindexes);
static void parallel_vacuum_index(Relation *Irel, IndexBulkDeleteResult **stats,
LVShared *lvshared, LVDeadTuples *dead_tuples,
- int nindexes, LVRelStats *vacrelstats);
+ int nindexes, LVSavedPosition *errinfo);
static void vacuum_indexes_leader(Relation *Irel, IndexBulkDeleteResult **stats,
LVRelStats *vacrelstats, LVParallelState *lps,
int nindexes);
static void vacuum_one_index(Relation indrel, IndexBulkDeleteResult **stats,
LVShared *lvshared, LVSharedIndStats *shared_indstats,
- LVDeadTuples *dead_tuples, LVRelStats *vacrelstats);
+ LVDeadTuples *dead_tuples, LVSavedPosition *errinfo);
static void lazy_cleanup_all_indexes(Relation *Irel, IndexBulkDeleteResult **stats,
LVRelStats *vacrelstats, LVParallelState *lps,
int nindexes);
@@ -1797,7 +1797,7 @@ lazy_vacuum_all_indexes(Relation onerel, Relation *Irel,
for (idx = 0; idx < nindexes; idx++)
lazy_vacuum_index(Irel[idx], &stats[idx], vacrelstats->dead_tuples,
- vacrelstats->old_live_tuples, vacrelstats);
+ vacrelstats->old_live_tuples, &vacrelstats->errinfo);
}
/* Increase and report the number of index scans */
@@ -2160,7 +2160,7 @@ lazy_parallel_vacuum_indexes(Relation *Irel, IndexBulkDeleteResult **stats,
* indexes in the case where no workers are launched.
*/
parallel_vacuum_index(Irel, stats, lps->lvshared,
- vacrelstats->dead_tuples, nindexes, vacrelstats);
+ vacrelstats->dead_tuples, nindexes, &vacrelstats->errinfo);
/*
* Next, accumulate buffer and WAL usage. (This must wait for the workers
@@ -2195,7 +2195,7 @@ lazy_parallel_vacuum_indexes(Relation *Irel, IndexBulkDeleteResult **stats,
static void
parallel_vacuum_index(Relation *Irel, IndexBulkDeleteResult **stats,
LVShared *lvshared, LVDeadTuples *dead_tuples,
- int nindexes, LVRelStats *vacrelstats)
+ int nindexes, LVSavedPosition *errinfo)
{
/*
* Increment the active worker count if we are able to launch any worker.
@@ -2229,7 +2229,7 @@ parallel_vacuum_index(Relation *Irel, IndexBulkDeleteResult **stats,
/* Do vacuum or cleanup of the index */
vacuum_one_index(Irel[idx], &(stats[idx]), lvshared, shared_indstats,
- dead_tuples, vacrelstats);
+ dead_tuples, errinfo);
}
/*
@@ -2270,7 +2270,7 @@ vacuum_indexes_leader(Relation *Irel, IndexBulkDeleteResult **stats,
skip_parallel_vacuum_index(Irel[i], lps->lvshared))
vacuum_one_index(Irel[i], &(stats[i]), lps->lvshared,
shared_indstats, vacrelstats->dead_tuples,
- vacrelstats);
+ &vacrelstats->errinfo);
}
/*
@@ -2290,7 +2290,7 @@ vacuum_indexes_leader(Relation *Irel, IndexBulkDeleteResult **stats,
static void
vacuum_one_index(Relation indrel, IndexBulkDeleteResult **stats,
LVShared *lvshared, LVSharedIndStats *shared_indstats,
- LVDeadTuples *dead_tuples, LVRelStats *vacrelstats)
+ LVDeadTuples *dead_tuples, LVSavedPosition *errinfo)
{
IndexBulkDeleteResult *bulkdelete_res = NULL;
@@ -2310,10 +2310,10 @@ vacuum_one_index(Relation indrel, IndexBulkDeleteResult **stats,
/* Do vacuum or cleanup of the index */
if (lvshared->for_cleanup)
lazy_cleanup_index(indrel, stats, lvshared->reltuples,
- lvshared->estimated_count, vacrelstats);
+ lvshared->estimated_count, errinfo);
else
lazy_vacuum_index(indrel, stats, dead_tuples,
- lvshared->reltuples, vacrelstats);
+ lvshared->reltuples, errinfo);
/*
* Copy the index bulk-deletion result returned from ambulkdelete and
@@ -2389,7 +2389,7 @@ lazy_cleanup_all_indexes(Relation *Irel, IndexBulkDeleteResult **stats,
lazy_cleanup_index(Irel[idx], &stats[idx],
vacrelstats->new_rel_tuples,
vacrelstats->tupcount_pages < vacrelstats->rel_pages,
- vacrelstats);
+ &vacrelstats->errinfo);
}
}
@@ -2404,7 +2404,7 @@ lazy_cleanup_all_indexes(Relation *Irel, IndexBulkDeleteResult **stats,
*/
static void
lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
- LVDeadTuples *dead_tuples, double reltuples, LVRelStats *vacrelstats)
+ LVDeadTuples *dead_tuples, double reltuples, LVSavedPosition *errinfo)
{
IndexVacuumInfo ivinfo;
const char *msg;
@@ -2422,10 +2422,10 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrinfo = vacrelstats->errinfo;
+ olderrinfo = *errinfo;
/* The index name is also saved during this phase */
- vacrelstats->errinfo.indname = pstrdup(RelationGetRelationName(indrel));
- update_vacuum_error_info(&vacrelstats->errinfo,
+ errinfo->indname = pstrdup(RelationGetRelationName(indrel));
+ update_vacuum_error_info(errinfo,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber);
@@ -2440,15 +2440,15 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ereport(elevel,
(errmsg(msg,
- vacrelstats->errinfo.indname,
+ errinfo->indname,
dead_tuples->num_tuples),
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(&vacrelstats->errinfo,
+ update_vacuum_error_info(errinfo,
olderrinfo.phase,
olderrinfo.blkno);
- pfree(vacrelstats->errinfo.indname);
+ pfree(errinfo->indname);
}
/*
@@ -2460,7 +2460,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
static void
lazy_cleanup_index(Relation indrel,
IndexBulkDeleteResult **stats,
- double reltuples, bool estimated_count, LVRelStats *vacrelstats)
+ double reltuples, bool estimated_count, LVSavedPosition *errinfo)
{
IndexVacuumInfo ivinfo;
const char *msg;
@@ -2479,20 +2479,20 @@ lazy_cleanup_index(Relation indrel,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrinfo = vacrelstats->errinfo;
+ olderrinfo = *errinfo;
/* The index name is also saved during this phase */
- vacrelstats->errinfo.indname = pstrdup(RelationGetRelationName(indrel));
- update_vacuum_error_info(&vacrelstats->errinfo,
+ errinfo->indname = pstrdup(RelationGetRelationName(indrel));
+ update_vacuum_error_info(errinfo,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber);
*stats = index_vacuum_cleanup(&ivinfo, *stats);
/* Revert back to the old phase information for error traceback */
- update_vacuum_error_info(&vacrelstats->errinfo,
+ update_vacuum_error_info(errinfo,
olderrinfo.phase,
olderrinfo.blkno);
- pfree(vacrelstats->errinfo.indname);
+ pfree(errinfo->indname);
if (!(*stats))
return;
@@ -3474,7 +3474,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
int nindexes;
char *sharedquery;
IndexBulkDeleteResult **stats;
- LVRelStats vacrelstats;
+ LVSavedPosition errinfo;
ErrorContextCallback errcallback;
lvshared = (LVShared *) shm_toc_lookup(toc, PARALLEL_VACUUM_KEY_SHARED,
@@ -3529,13 +3529,13 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
* Initialize vacrelstats for use as error callback arg by parallel
* worker.
*/
- vacrelstats.errinfo.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
- vacrelstats.errinfo.relname = pstrdup(RelationGetRelationName(onerel));
- vacrelstats.errinfo.phase = VACUUM_ERRCB_PHASE_UNKNOWN; /* Not yet processing */
+ errinfo.relnamespace = get_namespace_name(RelationGetNamespace(onerel));
+ errinfo.relname = pstrdup(RelationGetRelationName(onerel));
+ errinfo.phase = VACUUM_ERRCB_PHASE_UNKNOWN; /* Not yet processing */
/* Setup error traceback support for ereport() */
errcallback.callback = vacuum_error_callback;
- errcallback.arg = &vacrelstats.errinfo;
+ errcallback.arg = &errinfo;
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
@@ -3544,7 +3544,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
/* Process indexes to perform vacuum/cleanup */
parallel_vacuum_index(indrels, stats, lvshared, dead_tuples, nindexes,
- &vacrelstats);
+ &errinfo);
/* Report buffer/WAL usage during parallel execution */
buffer_usage = shm_toc_lookup(toc, PARALLEL_VACUUM_KEY_BUFFER_USAGE, false);
--
2.17.0
Hi,
On 2020-06-23 00:14:40 -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
I am only suggesting that where you save the old location, as currently
done with LVRelStats olderrinfo, you instead use a more specific
type. Not that you should pass that anywhere (except for
update_vacuum_error_info).As things currently stand, I don't think we need another struct
type at all. ISTM we should hard-wire the handling of indname
as I suggested above. Then there are only two fields to be dealt
with, and we could just as well save them in simple local variables.
That's fine with me too.
If there's a clear future path to needing to save/restore more
fields, then maybe another struct type would be useful ... but
right now the struct type declaration itself would take more
lines of code than it would save.
FWIW, I started to be annoyed by this code when I was addding
prefetching support to vacuum, and wanted to change what's tracked
where. The number of places that needed to be touched was
disproportional.
Here's a *draft* for how I thought this roughly could look like. I think
it's nicer to not specify the exact saved state in multiple places, and
I think it's much clearer to use a separate function to restore the
state than to set a "fresh" state.
I've only applied a hacky fix for the way the indname is tracked, I
thought that'd best be discussed separately.
Greetings,
Andres Freund
Attachments:
draft-vacuumlazy-errcontext-tracking.difftext/x-diff; charset=us-asciiDownload
diff --git i/src/backend/access/heap/vacuumlazy.c w/src/backend/access/heap/vacuumlazy.c
index 3bef0e124ba..afd6128233b 100644
--- i/src/backend/access/heap/vacuumlazy.c
+++ w/src/backend/access/heap/vacuumlazy.c
@@ -319,6 +319,12 @@ typedef struct LVRelStats
VacErrPhase phase;
} LVRelStats;
+typedef struct LVSavedPos
+{
+ BlockNumber blkno;
+ VacErrPhase phase;
+} LVSavedPos;
+
/* A few variables that don't seem worth passing around as parameters */
static int elevel = -1;
@@ -388,9 +394,9 @@ static void end_parallel_vacuum(Relation *Irel, IndexBulkDeleteResult **stats,
static LVSharedIndStats *get_indstats(LVShared *lvshared, int n);
static bool skip_parallel_vacuum_index(Relation indrel, LVShared *lvshared);
static void vacuum_error_callback(void *arg);
-static void update_vacuum_error_info(LVRelStats *errinfo, int phase,
- BlockNumber blkno, char *indname);
-
+static void update_vacuum_error_info(LVRelStats *errinfo, LVSavedPos *oldpos,
+ int phase, BlockNumber blkno, char *indname);
+static void restore_vacuum_error_info(LVRelStats *errinfo, const LVSavedPos *oldpos);
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -538,7 +544,8 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
* which we add context information to errors, so we don't need to
* revert to the previous phase.
*/
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_TRUNCATE,
+ update_vacuum_error_info(vacrelstats, NULL,
+ VACUUM_ERRCB_PHASE_TRUNCATE,
vacrelstats->nonempty_pages, NULL);
lazy_truncate_heap(onerel, vacrelstats);
}
@@ -948,8 +955,8 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_SCANNED, blkno);
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_SCAN_HEAP,
- blkno, NULL);
+ update_vacuum_error_info(vacrelstats, NULL,
+ VACUUM_ERRCB_PHASE_SCAN_HEAP, blkno, NULL);
if (blkno == next_unskippable_block)
{
@@ -1820,15 +1827,14 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
int npages;
PGRUsage ru0;
Buffer vmbuffer = InvalidBuffer;
- LVRelStats olderrinfo;
+ LVSavedPos oldpos;
/* Report that we are now vacuuming the heap */
pgstat_progress_update_param(PROGRESS_VACUUM_PHASE,
PROGRESS_VACUUM_PHASE_VACUUM_HEAP);
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ update_vacuum_error_info(vacrelstats, &oldpos, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
InvalidBlockNumber, NULL);
pg_rusage_init(&ru0);
@@ -1879,10 +1885,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ restore_vacuum_error_info(vacrelstats, &oldpos);
}
/*
@@ -1905,14 +1908,13 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
- LVRelStats olderrinfo;
+ LVSavedPos oldpos;
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
- blkno, NULL);
+ update_vacuum_error_info(vacrelstats, &oldpos,
+ VACUUM_ERRCB_PHASE_VACUUM_HEAP, blkno, NULL);
START_CRIT_SECTION();
@@ -1991,10 +1993,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
}
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ restore_vacuum_error_info(vacrelstats, &oldpos);
return tupindex;
}
@@ -2404,7 +2403,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrinfo;
+ LVSavedPos oldpos;
pg_rusage_init(&ru0);
@@ -2417,8 +2416,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats,
+ update_vacuum_error_info(vacrelstats, &oldpos,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber,
RelationGetRelationName(indrel));
@@ -2439,10 +2437,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ restore_vacuum_error_info(vacrelstats, &oldpos);
}
/*
@@ -2459,7 +2454,7 @@ lazy_cleanup_index(Relation indrel,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrcbarg;
+ LVSavedPos oldpos;
pg_rusage_init(&ru0);
@@ -2473,8 +2468,7 @@ lazy_cleanup_index(Relation indrel,
ivinfo.strategy = vac_strategy;
/* Update error traceback information */
- olderrcbarg = *vacrelstats;
- update_vacuum_error_info(vacrelstats,
+ update_vacuum_error_info(vacrelstats, &oldpos,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber,
RelationGetRelationName(indrel));
@@ -2482,10 +2476,8 @@ lazy_cleanup_index(Relation indrel,
*stats = index_vacuum_cleanup(&ivinfo, *stats);
/* Revert back to the old phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrcbarg.phase,
- olderrcbarg.blkno,
- olderrcbarg.indname);
+ restore_vacuum_error_info(vacrelstats, &oldpos);
+
if (!(*stats))
return;
@@ -3600,9 +3592,15 @@ vacuum_error_callback(void *arg)
/* Update vacuum error callback for the current phase, block, and index. */
static void
-update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno,
- char *indname)
+update_vacuum_error_info(LVRelStats *errinfo, LVSavedPos *oldpos,
+ int phase, BlockNumber blkno, char *indname)
{
+ if (oldpos)
+ {
+ oldpos->blkno = errinfo->blkno;
+ oldpos->phase = errinfo->phase;
+ }
+
errinfo->blkno = blkno;
errinfo->phase = phase;
@@ -3613,3 +3611,17 @@ update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno,
/* For index phases, save the name of the current index for the callback */
errinfo->indname = indname ? pstrdup(indname) : NULL;
}
+
+static void
+restore_vacuum_error_info(LVRelStats *errinfo, const LVSavedPos *oldpos)
+{
+ errinfo->blkno = oldpos->blkno;
+ errinfo->phase = oldpos->phase;
+
+ /* FIXME: Imo pretty ugly that we don't restore the name properly */
+ if (errinfo->indname)
+ {
+ pfree(errinfo->indname);
+ errinfo->indname = NULL;
+ }
+}
On Tue, Jun 23, 2020 at 11:49 PM Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2020-06-23 00:14:40 -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
I am only suggesting that where you save the old location, as currently
done with LVRelStats olderrinfo, you instead use a more specific
type. Not that you should pass that anywhere (except for
update_vacuum_error_info).As things currently stand, I don't think we need another struct
type at all. ISTM we should hard-wire the handling of indname
as I suggested above. Then there are only two fields to be dealt
with, and we could just as well save them in simple local variables.That's fine with me too.
I have looked at both the patches (using separate variables (by
Justin) and using a struct (by Andres)) and found the second one bit
better.
If there's a clear future path to needing to save/restore more
fields, then maybe another struct type would be useful ... but
right now the struct type declaration itself would take more
lines of code than it would save.FWIW, I started to be annoyed by this code when I was addding
prefetching support to vacuum, and wanted to change what's tracked
where. The number of places that needed to be touched was
disproportional.Here's a *draft* for how I thought this roughly could look like. I think
it's nicer to not specify the exact saved state in multiple places, and
I think it's much clearer to use a separate function to restore the
state than to set a "fresh" state.
I think this is a good idea and makes code look better. I think it is
better to name new struct as LVSavedErrInfo instead of LVSavedPos.
I've only applied a hacky fix for the way the indname is tracked, I
thought that'd best be discussed separately.
I think it is better to use Tom's idea here to save and restore index
information in-place. I have used Justin's patch with some
improvements like adding Asserts and initializing with NULL for
indname while restoring to make things unambiguous.
I have improved some comments in the code and for now, kept as two
patches (a) one for improving the error info for index (mostly
Justin's patch based on Tom's idea) and (b) the other to generally
improve the code in this area (mostly Andres's patch).
I have done some testing with both the patches and would like to do
more unless there are objections with these.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
improve_vac_err_cb_v1.patchapplication/octet-stream; name=improve_vac_err_cb_v1.patchDownload
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3bef0e1..58bc3df 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -389,7 +389,7 @@ static LVSharedIndStats *get_indstats(LVShared *lvshared, int n);
static bool skip_parallel_vacuum_index(Relation indrel, LVShared *lvshared);
static void vacuum_error_callback(void *arg);
static void update_vacuum_error_info(LVRelStats *errinfo, int phase,
- BlockNumber blkno, char *indname);
+ BlockNumber blkno);
/*
@@ -539,7 +539,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
* revert to the previous phase.
*/
update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_TRUNCATE,
- vacrelstats->nonempty_pages, NULL);
+ vacrelstats->nonempty_pages);
lazy_truncate_heap(onerel, vacrelstats);
}
@@ -949,7 +949,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_SCANNED, blkno);
update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_SCAN_HEAP,
- blkno, NULL);
+ blkno);
if (blkno == next_unskippable_block)
{
@@ -1829,7 +1829,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
/* Update error traceback information */
olderrinfo = *vacrelstats;
update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
- InvalidBlockNumber, NULL);
+ InvalidBlockNumber);
pg_rusage_init(&ru0);
npages = 0;
@@ -1881,8 +1881,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
/* Revert to the previous phase information for error traceback */
update_vacuum_error_info(vacrelstats,
olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ olderrinfo.blkno);
}
/*
@@ -1912,7 +1911,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
/* Update error traceback information */
olderrinfo = *vacrelstats;
update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
- blkno, NULL);
+ blkno);
START_CRIT_SECTION();
@@ -1993,8 +1992,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
/* Revert to the previous phase information for error traceback */
update_vacuum_error_info(vacrelstats,
olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ olderrinfo.blkno);
return tupindex;
}
@@ -2418,10 +2416,16 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
/* Update error traceback information */
olderrinfo = *vacrelstats;
+
+ /*
+ * The index name is saved only during this phase and restored immediately
+ * after this phase. See vacuum_error_callback.
+ */
+ Assert(vacrelstats->indname == NULL);
+ vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
update_vacuum_error_info(vacrelstats,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
- InvalidBlockNumber,
- RelationGetRelationName(indrel));
+ InvalidBlockNumber);
/* Do bulk deletion */
*stats = index_bulk_delete(&ivinfo, *stats,
@@ -2441,8 +2445,9 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
/* Revert to the previous phase information for error traceback */
update_vacuum_error_info(vacrelstats,
olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ olderrinfo.blkno);
+ pfree(vacrelstats->indname);
+ vacrelstats->indname = NULL;
}
/*
@@ -2474,18 +2479,26 @@ lazy_cleanup_index(Relation indrel,
/* Update error traceback information */
olderrcbarg = *vacrelstats;
+
+ /*
+ * The index name is saved only during this phase and restored immediately
+ * after this phase. See vacuum_error_callback.
+ */
+ Assert(vacrelstats->indname == NULL);
+ vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
update_vacuum_error_info(vacrelstats,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
- InvalidBlockNumber,
- RelationGetRelationName(indrel));
+ InvalidBlockNumber);
*stats = index_vacuum_cleanup(&ivinfo, *stats);
/* Revert back to the old phase information for error traceback */
update_vacuum_error_info(vacrelstats,
olderrcbarg.phase,
- olderrcbarg.blkno,
- olderrcbarg.indname);
+ olderrcbarg.blkno);
+ pfree(vacrelstats->indname);
+ vacrelstats->indname = NULL;
+
if (!(*stats))
return;
@@ -3598,18 +3611,10 @@ vacuum_error_callback(void *arg)
}
}
-/* Update vacuum error callback for the current phase, block, and index. */
+/* Update vacuum error callback for the current phase and block. */
static void
-update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno,
- char *indname)
+update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno)
{
errinfo->blkno = blkno;
errinfo->phase = phase;
-
- /* Free index name from any previous phase */
- if (errinfo->indname)
- pfree(errinfo->indname);
-
- /* For index phases, save the name of the current index for the callback */
- errinfo->indname = indname ? pstrdup(indname) : NULL;
}
improve_vac_err_cb_v2.patchapplication/octet-stream; name=improve_vac_err_cb_v2.patchDownload
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 58bc3df..39a0fdb 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -319,6 +319,13 @@ typedef struct LVRelStats
VacErrPhase phase;
} LVRelStats;
+/* Struct for saving and restoring vacuum error information. */
+typedef struct LVSavedErrInfo
+{
+ BlockNumber blkno;
+ VacErrPhase phase;
+} LVSavedErrInfo;
+
/* A few variables that don't seem worth passing around as parameters */
static int elevel = -1;
@@ -388,8 +395,9 @@ static void end_parallel_vacuum(Relation *Irel, IndexBulkDeleteResult **stats,
static LVSharedIndStats *get_indstats(LVShared *lvshared, int n);
static bool skip_parallel_vacuum_index(Relation indrel, LVShared *lvshared);
static void vacuum_error_callback(void *arg);
-static void update_vacuum_error_info(LVRelStats *errinfo, int phase,
- BlockNumber blkno);
+static void update_vacuum_error_info(LVRelStats *errinfo, LVSavedErrInfo *oldpos,
+ int phase, BlockNumber blkno);
+static void restore_vacuum_error_info(LVRelStats *errinfo, const LVSavedErrInfo *oldpos);
/*
@@ -538,7 +546,7 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
* which we add context information to errors, so we don't need to
* revert to the previous phase.
*/
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_TRUNCATE,
+ update_vacuum_error_info(vacrelstats, NULL, VACUUM_ERRCB_PHASE_TRUNCATE,
vacrelstats->nonempty_pages);
lazy_truncate_heap(onerel, vacrelstats);
}
@@ -948,7 +956,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_SCANNED, blkno);
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_SCAN_HEAP,
+ update_vacuum_error_info(vacrelstats, NULL, VACUUM_ERRCB_PHASE_SCAN_HEAP,
blkno);
if (blkno == next_unskippable_block)
@@ -1820,15 +1828,14 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
int npages;
PGRUsage ru0;
Buffer vmbuffer = InvalidBuffer;
- LVRelStats olderrinfo;
+ LVSavedErrInfo oldpos;
/* Report that we are now vacuuming the heap */
pgstat_progress_update_param(PROGRESS_VACUUM_PHASE,
PROGRESS_VACUUM_PHASE_VACUUM_HEAP);
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ update_vacuum_error_info(vacrelstats, &oldpos, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
InvalidBlockNumber);
pg_rusage_init(&ru0);
@@ -1879,9 +1886,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno);
+ restore_vacuum_error_info(vacrelstats, &oldpos);
}
/*
@@ -1904,13 +1909,12 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
- LVRelStats olderrinfo;
+ LVSavedErrInfo oldpos;
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ update_vacuum_error_info(vacrelstats, &oldpos, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
blkno);
START_CRIT_SECTION();
@@ -1990,9 +1994,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
}
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno);
+ restore_vacuum_error_info(vacrelstats, &oldpos);
return tupindex;
}
@@ -2402,7 +2404,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrinfo;
+ LVSavedErrInfo oldpos;
pg_rusage_init(&ru0);
@@ -2414,16 +2416,15 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ivinfo.num_heap_tuples = reltuples;
ivinfo.strategy = vac_strategy;
- /* Update error traceback information */
- olderrinfo = *vacrelstats;
-
/*
+ * Update error traceback information.
+ *
* The index name is saved only during this phase and restored immediately
* after this phase. See vacuum_error_callback.
*/
Assert(vacrelstats->indname == NULL);
vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
- update_vacuum_error_info(vacrelstats,
+ update_vacuum_error_info(vacrelstats, &oldpos,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber);
@@ -2443,9 +2444,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno);
+ restore_vacuum_error_info(vacrelstats, &oldpos);
pfree(vacrelstats->indname);
vacrelstats->indname = NULL;
}
@@ -2464,7 +2463,7 @@ lazy_cleanup_index(Relation indrel,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrcbarg;
+ LVSavedErrInfo oldpos;
pg_rusage_init(&ru0);
@@ -2477,25 +2476,22 @@ lazy_cleanup_index(Relation indrel,
ivinfo.num_heap_tuples = reltuples;
ivinfo.strategy = vac_strategy;
- /* Update error traceback information */
- olderrcbarg = *vacrelstats;
-
/*
+ * Update error traceback information.
+ *
* The index name is saved only during this phase and restored immediately
* after this phase. See vacuum_error_callback.
*/
Assert(vacrelstats->indname == NULL);
vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
- update_vacuum_error_info(vacrelstats,
+ update_vacuum_error_info(vacrelstats, &oldpos,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber);
*stats = index_vacuum_cleanup(&ivinfo, *stats);
/* Revert back to the old phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrcbarg.phase,
- olderrcbarg.blkno);
+ restore_vacuum_error_info(vacrelstats, &oldpos);
pfree(vacrelstats->indname);
vacrelstats->indname = NULL;
@@ -3611,10 +3607,30 @@ vacuum_error_callback(void *arg)
}
}
-/* Update vacuum error callback for the current phase and block. */
+/*
+ * Updates the information required for vacuum error callback. This also saves
+ * the current information which can be later restored via restore_vacuum_error_info.
+ */
static void
-update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno)
+update_vacuum_error_info(LVRelStats *errinfo, LVSavedErrInfo *oldpos, int phase,
+ BlockNumber blkno)
{
+ if (oldpos)
+ {
+ oldpos->blkno = errinfo->blkno;
+ oldpos->phase = errinfo->phase;
+ }
+
errinfo->blkno = blkno;
errinfo->phase = phase;
}
+
+/*
+ * Restores the vacuum information saved via a prior call to update_vacuum_error_info.
+ */
+static void
+restore_vacuum_error_info(LVRelStats *errinfo, const LVSavedErrInfo *oldpos)
+{
+ errinfo->blkno = oldpos->blkno;
+ errinfo->phase = oldpos->phase;
+}
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c65a552..7eaaad1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1250,6 +1250,7 @@ LUID
LVDeadTuples
LVParallelState
LVRelStats
+LVSavedErrInfo
LVShared
LVSharedIndStats
LWLock
On Thu, Jun 25, 2020 at 02:31:51PM +0530, Amit Kapila wrote:
I have looked at both the patches (using separate variables (by
Justin) and using a struct (by Andres)) and found the second one bit
better.
Thanks for looking.
I have improved some comments in the code and for now, kept as two
patches (a) one for improving the error info for index (mostly
Justin's patch based on Tom's idea) and (b) the other to generally
improve the code in this area (mostly Andres's patch).
And thanks for separate patchen :)
I have done some testing with both the patches and would like to do
more unless there are objections with these.
Comments:
* The index name is saved only during this phase and restored immediately
=> I wouldn't say "only" since it's saved during lazy_vacuum: index AND cleanup.
update_vacuum_error_info(LVRelStats *errinfo, LVSavedErrInfo *oldpos, int phase,
=> You called your struct "LVSavedErrInfo" but the variables are still called
"pos". I would call it olderrinfo or just old.
Also, this doesn't (re)rename the "cbarg" stuff that Alvaro didn't like, which
was my 0001 patch.
--
Justin
On Fri, Jun 26, 2020 at 7:25 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
I have done some testing with both the patches and would like to do
more unless there are objections with these.Comments:
* The index name is saved only during this phase and restored immediately
=> I wouldn't say "only" since it's saved during lazy_vacuum: index AND cleanup.
update_vacuum_error_info(LVRelStats *errinfo, LVSavedErrInfo *oldpos, int phase,
=> You called your struct "LVSavedErrInfo" but the variables are still called
"pos". I would call it olderrinfo or just old.
Fixed both of the above comments. I used the variable name as saved_err_info.
Also, this doesn't (re)rename the "cbarg" stuff that Alvaro didn't like, which
was my 0001 patch.
If I am not missing anything then that change was in
lazy_cleanup_index and after this patch, it won't be required because
we are using a different variable name.
I have combined both the patches now.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
0001-Improve-vacuum-error-context-handling.v1.patchapplication/octet-stream; name=0001-Improve-vacuum-error-context-handling.v1.patchDownload
From 89d883387fadb318c3b93b6208acc34cae69111a Mon Sep 17 00:00:00 2001
From: Amit Kapila <akapila@postgresql.org>
Date: Fri, 26 Jun 2020 09:01:38 +0530
Subject: [PATCH] Improve vacuum error context handling.
Use separate functions to save and restore error context information as
that made code easier to understand. Also, make it clear that the index
information required for error context is sane.
Author: Andres Freund, Justin Pryzby, Amit Kapila
Backpatch-through: 13, where it was introduced
Discussion: https://postgr.es/m/CAA4eK1LWo+v1OWu=Sky27GTGSCuOmr7iaURNbc5xz6jO+SaPeA@mail.gmail.com
---
src/backend/access/heap/vacuumlazy.c | 123 ++++++++++++++++++++---------------
src/tools/pgindent/typedefs.list | 1 +
2 files changed, 73 insertions(+), 51 deletions(-)
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 3bef0e1..68effca 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -319,6 +319,13 @@ typedef struct LVRelStats
VacErrPhase phase;
} LVRelStats;
+/* Struct for saving and restoring vacuum error information. */
+typedef struct LVSavedErrInfo
+{
+ BlockNumber blkno;
+ VacErrPhase phase;
+} LVSavedErrInfo;
+
/* A few variables that don't seem worth passing around as parameters */
static int elevel = -1;
@@ -388,8 +395,9 @@ static void end_parallel_vacuum(Relation *Irel, IndexBulkDeleteResult **stats,
static LVSharedIndStats *get_indstats(LVShared *lvshared, int n);
static bool skip_parallel_vacuum_index(Relation indrel, LVShared *lvshared);
static void vacuum_error_callback(void *arg);
-static void update_vacuum_error_info(LVRelStats *errinfo, int phase,
- BlockNumber blkno, char *indname);
+static void update_vacuum_error_info(LVRelStats *errinfo, LVSavedErrInfo *saved_err_info,
+ int phase, BlockNumber blkno);
+static void restore_vacuum_error_info(LVRelStats *errinfo, const LVSavedErrInfo *saved_err_info);
/*
@@ -538,8 +546,8 @@ heap_vacuum_rel(Relation onerel, VacuumParams *params,
* which we add context information to errors, so we don't need to
* revert to the previous phase.
*/
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_TRUNCATE,
- vacrelstats->nonempty_pages, NULL);
+ update_vacuum_error_info(vacrelstats, NULL, VACUUM_ERRCB_PHASE_TRUNCATE,
+ vacrelstats->nonempty_pages);
lazy_truncate_heap(onerel, vacrelstats);
}
@@ -948,8 +956,8 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats,
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_SCANNED, blkno);
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_SCAN_HEAP,
- blkno, NULL);
+ update_vacuum_error_info(vacrelstats, NULL, VACUUM_ERRCB_PHASE_SCAN_HEAP,
+ blkno);
if (blkno == next_unskippable_block)
{
@@ -1820,16 +1828,15 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
int npages;
PGRUsage ru0;
Buffer vmbuffer = InvalidBuffer;
- LVRelStats olderrinfo;
+ LVSavedErrInfo saved_err_info;
/* Report that we are now vacuuming the heap */
pgstat_progress_update_param(PROGRESS_VACUUM_PHASE,
PROGRESS_VACUUM_PHASE_VACUUM_HEAP);
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
- InvalidBlockNumber, NULL);
+ update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ InvalidBlockNumber);
pg_rusage_init(&ru0);
npages = 0;
@@ -1879,10 +1886,7 @@ lazy_vacuum_heap(Relation onerel, LVRelStats *vacrelstats)
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ restore_vacuum_error_info(vacrelstats, &saved_err_info);
}
/*
@@ -1905,14 +1909,13 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
int uncnt = 0;
TransactionId visibility_cutoff_xid;
bool all_frozen;
- LVRelStats olderrinfo;
+ LVSavedErrInfo saved_err_info;
pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno);
/* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
- blkno, NULL);
+ update_vacuum_error_info(vacrelstats, &saved_err_info, VACUUM_ERRCB_PHASE_VACUUM_HEAP,
+ blkno);
START_CRIT_SECTION();
@@ -1991,10 +1994,7 @@ lazy_vacuum_page(Relation onerel, BlockNumber blkno, Buffer buffer,
}
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ restore_vacuum_error_info(vacrelstats, &saved_err_info);
return tupindex;
}
@@ -2404,7 +2404,7 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrinfo;
+ LVSavedErrInfo saved_err_info;
pg_rusage_init(&ru0);
@@ -2416,12 +2416,17 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
ivinfo.num_heap_tuples = reltuples;
ivinfo.strategy = vac_strategy;
- /* Update error traceback information */
- olderrinfo = *vacrelstats;
- update_vacuum_error_info(vacrelstats,
+ /*
+ * Update error traceback information.
+ *
+ * The index name is saved during this phase and restored immediately
+ * after this phase. See vacuum_error_callback.
+ */
+ Assert(vacrelstats->indname == NULL);
+ vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
+ update_vacuum_error_info(vacrelstats, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
- InvalidBlockNumber,
- RelationGetRelationName(indrel));
+ InvalidBlockNumber);
/* Do bulk deletion */
*stats = index_bulk_delete(&ivinfo, *stats,
@@ -2439,10 +2444,9 @@ lazy_vacuum_index(Relation indrel, IndexBulkDeleteResult **stats,
errdetail_internal("%s", pg_rusage_show(&ru0))));
/* Revert to the previous phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrinfo.phase,
- olderrinfo.blkno,
- olderrinfo.indname);
+ restore_vacuum_error_info(vacrelstats, &saved_err_info);
+ pfree(vacrelstats->indname);
+ vacrelstats->indname = NULL;
}
/*
@@ -2459,7 +2463,7 @@ lazy_cleanup_index(Relation indrel,
IndexVacuumInfo ivinfo;
const char *msg;
PGRUsage ru0;
- LVRelStats olderrcbarg;
+ LVSavedErrInfo saved_err_info;
pg_rusage_init(&ru0);
@@ -2472,20 +2476,25 @@ lazy_cleanup_index(Relation indrel,
ivinfo.num_heap_tuples = reltuples;
ivinfo.strategy = vac_strategy;
- /* Update error traceback information */
- olderrcbarg = *vacrelstats;
- update_vacuum_error_info(vacrelstats,
+ /*
+ * Update error traceback information.
+ *
+ * The index name is saved during this phase and restored immediately
+ * after this phase. See vacuum_error_callback.
+ */
+ Assert(vacrelstats->indname == NULL);
+ vacrelstats->indname = pstrdup(RelationGetRelationName(indrel));
+ update_vacuum_error_info(vacrelstats, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
- InvalidBlockNumber,
- RelationGetRelationName(indrel));
+ InvalidBlockNumber);
*stats = index_vacuum_cleanup(&ivinfo, *stats);
/* Revert back to the old phase information for error traceback */
- update_vacuum_error_info(vacrelstats,
- olderrcbarg.phase,
- olderrcbarg.blkno,
- olderrcbarg.indname);
+ restore_vacuum_error_info(vacrelstats, &saved_err_info);
+ pfree(vacrelstats->indname);
+ vacrelstats->indname = NULL;
+
if (!(*stats))
return;
@@ -3598,18 +3607,30 @@ vacuum_error_callback(void *arg)
}
}
-/* Update vacuum error callback for the current phase, block, and index. */
+/*
+ * Updates the information required for vacuum error callback. This also saves
+ * the current information which can be later restored via restore_vacuum_error_info.
+ */
static void
-update_vacuum_error_info(LVRelStats *errinfo, int phase, BlockNumber blkno,
- char *indname)
+update_vacuum_error_info(LVRelStats *errinfo, LVSavedErrInfo *saved_err_info, int phase,
+ BlockNumber blkno)
{
+ if (saved_err_info)
+ {
+ saved_err_info->blkno = errinfo->blkno;
+ saved_err_info->phase = errinfo->phase;
+ }
+
errinfo->blkno = blkno;
errinfo->phase = phase;
+}
- /* Free index name from any previous phase */
- if (errinfo->indname)
- pfree(errinfo->indname);
-
- /* For index phases, save the name of the current index for the callback */
- errinfo->indname = indname ? pstrdup(indname) : NULL;
+/*
+ * Restores the vacuum information saved via a prior call to update_vacuum_error_info.
+ */
+static void
+restore_vacuum_error_info(LVRelStats *errinfo, const LVSavedErrInfo *saved_err_info)
+{
+ errinfo->blkno = saved_err_info->blkno;
+ errinfo->phase = saved_err_info->phase;
}
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c65a552..7eaaad1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1250,6 +1250,7 @@ LUID
LVDeadTuples
LVParallelState
LVRelStats
+LVSavedErrInfo
LVShared
LVSharedIndStats
LWLock
--
1.8.3.1
On Fri, Jun 26, 2020 at 9:19 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Fri, Jun 26, 2020 at 7:25 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Comments:
* The index name is saved only during this phase and restored immediately
=> I wouldn't say "only" since it's saved during lazy_vacuum: index AND cleanup.
update_vacuum_error_info(LVRelStats *errinfo, LVSavedErrInfo *oldpos, int phase,
=> You called your struct "LVSavedErrInfo" but the variables are still called
"pos". I would call it olderrinfo or just old.Fixed both of the above comments. I used the variable name as saved_err_info.
Also, this doesn't (re)rename the "cbarg" stuff that Alvaro didn't like, which
was my 0001 patch.If I am not missing anything then that change was in
lazy_cleanup_index and after this patch, it won't be required because
we are using a different variable name.I have combined both the patches now.
I am planning to push this tomorrow if there are no further
suggestions/comments.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
On Tue, Jun 30, 2020 at 9:30 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
If I am not missing anything then that change was in
lazy_cleanup_index and after this patch, it won't be required because
we are using a different variable name.I have combined both the patches now.
I am planning to push this tomorrow if there are no further
suggestions/comments.
Pushed. Now, coming back to the question of the back patch. I see a
point in deferring this for 3-6 months or maybe more after PG13 is
released. OTOH, this implementation is mainly triggered by issues
reported in this area and this doesn't seem to be a very invasive
patch which can cause some de-stabilization in back-branches. I am not
in a hurry to get this backpatched but still, it would be good if this
can be backpatched earlier as quite a few people (onlist and EDB
customers) have reported issues that could have been narrowed down if
this patch is present in back-branches.
It seems Alvaro and I are in favor of backpatch whereas Andres and
Justin seem to think it should be deferred until this change has seen
some real-world exposure.
Anyone else wants to weigh in?
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
On Thu, Jul 2, 2020 at 9:07 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Tue, Jun 30, 2020 at 9:30 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
If I am not missing anything then that change was in
lazy_cleanup_index and after this patch, it won't be required because
we are using a different variable name.I have combined both the patches now.
I am planning to push this tomorrow if there are no further
suggestions/comments.Pushed. Now, coming back to the question of the back patch. I see a
point in deferring this for 3-6 months or maybe more after PG13 is
released. OTOH, this implementation is mainly triggered by issues
reported in this area and this doesn't seem to be a very invasive
patch which can cause some de-stabilization in back-branches. I am not
in a hurry to get this backpatched but still, it would be good if this
can be backpatched earlier as quite a few people (onlist and EDB
customers) have reported issues that could have been narrowed down if
this patch is present in back-branches.It seems Alvaro and I are in favor of backpatch whereas Andres and
Justin seem to think it should be deferred until this change has seen
some real-world exposure.Anyone else wants to weigh in?
Seeing no more responses, it seems better to defer this backpatch till
PG13 is out and we get some confidence in this functionality.
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com