add PROCESS_MAIN to VACUUM
Hi hackers,
I originally suggested $ЅUBJECT as part of the thread that ultimately led
to the addition of PROCESS_TOAST [0]/messages/by-id/BA8951E9-1524-48C5-94AF-73B1F0D7857F@amazon.com, but we decided not to proceed with
it. Recently, this idea came up again [1]/messages/by-id/20221215191246.GA252861@nathanxps13, so I thought I'd give it
another try.
The motivation for adding this option is to make it easier to VACUUM only a
relation's TOAST table. At the moment, you need to find the TOAST table by
examining a relation's reltoastrelid, and you need USAGE on the pg_toast
schema. This option could also help make it possible to call only
vac_update_datfrozenxid() without processing any relations, as discussed
elsewhere [2]/messages/by-id/20221229213719.GA301584@nathanxps13.
The demand for all these niche VACUUM options is likely limited, but it
does seem like there are some useful applications. If a new option is out
of the question, perhaps this functionality could be added to the existing
PROCESS_TOAST option.
[0]: /messages/by-id/BA8951E9-1524-48C5-94AF-73B1F0D7857F@amazon.com
[1]: /messages/by-id/20221215191246.GA252861@nathanxps13
[2]: /messages/by-id/20221229213719.GA301584@nathanxps13
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Attachments:
v1-0001-add-PROCESS_MAIN-to-VACUUM.patchtext/x-diff; charset=us-asciiDownload
From b123ddaf560a647536f5a7e52993401755da8650 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathandbossart@gmail.com>
Date: Thu, 29 Dec 2022 15:31:49 -0800
Subject: [PATCH v1 1/1] add PROCESS_MAIN to VACUUM
---
doc/src/sgml/ref/vacuum.sgml | 13 +++++++++++++
doc/src/sgml/ref/vacuumdb.sgml | 15 +++++++++++++++
src/backend/commands/vacuum.c | 24 +++++++++++++++++++-----
src/backend/postmaster/autovacuum.c | 1 +
src/bin/psql/tab-complete.c | 4 ++--
src/bin/scripts/t/100_vacuumdb.pl | 7 +++++++
src/bin/scripts/vacuumdb.c | 24 ++++++++++++++++++++++++
src/include/commands/vacuum.h | 1 +
src/test/regress/expected/vacuum.out | 4 ++++
src/test/regress/sql/vacuum.sql | 5 +++++
10 files changed, 91 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index e14ead8826..bb634ae569 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -33,6 +33,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
DISABLE_PAGE_SKIPPING [ <replaceable class="parameter">boolean</replaceable> ]
SKIP_LOCKED [ <replaceable class="parameter">boolean</replaceable> ]
INDEX_CLEANUP { AUTO | ON | OFF }
+ PROCESS_MAIN [ <replaceable class="parameter">boolean</replaceable> ]
PROCESS_TOAST [ <replaceable class="parameter">boolean</replaceable> ]
TRUNCATE [ <replaceable class="parameter">boolean</replaceable> ]
PARALLEL <replaceable class="parameter">integer</replaceable>
@@ -236,6 +237,18 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>PROCESS_MAIN</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should attempt to process the
+ main relation. This is usually the desired behavior and is the default.
+ Setting this option to false may be useful when it is only necessary to
+ vacuum a relation's corresponding <literal>TOAST</literal> table.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>PROCESS_TOAST</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/vacuumdb.sgml b/doc/src/sgml/ref/vacuumdb.sgml
index 841aced3bd..74bac2d4ba 100644
--- a/doc/src/sgml/ref/vacuumdb.sgml
+++ b/doc/src/sgml/ref/vacuumdb.sgml
@@ -317,6 +317,21 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--no-process-main</option></term>
+ <listitem>
+ <para>
+ Skip the main relation.
+ </para>
+ <note>
+ <para>
+ This option is only available for servers running
+ <productname>PostgreSQL</productname> 16 and later.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--no-process-toast</option></term>
<listitem>
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ba965b8c7b..4dbad90039 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -113,6 +113,7 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
bool freeze = false;
bool full = false;
bool disable_page_skipping = false;
+ bool process_main = true;
bool process_toast = true;
ListCell *lc;
@@ -164,6 +165,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
params.index_cleanup = get_vacoptval_from_boolean(opt);
}
}
+ else if (strcmp(opt->defname, "process_main") == 0)
+ process_main = defGetBoolean(opt);
else if (strcmp(opt->defname, "process_toast") == 0)
process_toast = defGetBoolean(opt);
else if (strcmp(opt->defname, "truncate") == 0)
@@ -216,7 +219,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
(freeze ? VACOPT_FREEZE : 0) |
(full ? VACOPT_FULL : 0) |
(disable_page_skipping ? VACOPT_DISABLE_PAGE_SKIPPING : 0) |
- (process_toast ? VACOPT_PROCESS_TOAST : 0);
+ (process_toast ? VACOPT_PROCESS_TOAST : 0) |
+ (process_main ? VACOPT_PROCESS_MAIN : 0);
/* sanity checks on options */
Assert(params.options & (VACOPT_VACUUM | VACOPT_ANALYZE));
@@ -1994,10 +1998,12 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params)
/*
* Remember the relation's TOAST relation for later, if the caller asked
* us to process it. In VACUUM FULL, though, the toast table is
- * automatically rebuilt by cluster_rel so we shouldn't recurse to it.
+ * automatically rebuilt by cluster_rel so we shouldn't recurse to it
+ * unless PROCESS_MAIN is disabled.
*/
if ((params->options & VACOPT_PROCESS_TOAST) != 0 &&
- (params->options & VACOPT_FULL) == 0)
+ ((params->options & VACOPT_FULL) == 0 ||
+ (params->options & VACOPT_PROCESS_MAIN) == 0))
toast_relid = rel->rd_rel->reltoastrelid;
else
toast_relid = InvalidOid;
@@ -2016,7 +2022,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params)
/*
* Do the actual work --- either FULL or "lazy" vacuum
*/
- if (params->options & VACOPT_FULL)
+ if (params->options & VACOPT_FULL &&
+ params->options & VACOPT_PROCESS_MAIN)
{
ClusterParams cluster_params = {0};
@@ -2030,7 +2037,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params)
/* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
cluster_rel(relid, InvalidOid, &cluster_params);
}
- else
+ else if (params->options & VACOPT_PROCESS_MAIN)
table_relation_vacuum(rel, params, vac_strategy);
/* Roll back any GUC changes executed by index functions */
@@ -2057,7 +2064,14 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params)
* totally unimportant for toast relations.
*/
if (toast_relid != InvalidOid)
+ {
+ /* we force VACOPT_PROCESS_MAIN so vacuum_rel() processes it */
+ bool force_opt = ((params->options & VACOPT_PROCESS_MAIN) == 0);
+
+ params->options |= force_opt ? VACOPT_PROCESS_MAIN : 0;
vacuum_rel(toast_relid, NULL, params);
+ params->options &= force_opt ? ~VACOPT_PROCESS_MAIN : ~0;
+ }
/*
* Now release the session-level lock on the main table.
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 0746d80224..65c6fb6a27 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -2856,6 +2856,7 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
/* Note that this skips toast relations */
tab->at_params.options = (dovacuum ? VACOPT_VACUUM : 0) |
+ (dovacuum ? VACOPT_PROCESS_MAIN : 0) |
(doanalyze ? VACOPT_ANALYZE : 0) |
(!wraparound ? VACOPT_SKIP_LOCKED : 0);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 2a3921937c..ce4c16c8a3 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -4551,9 +4551,9 @@ psql_completion(const char *text, int start, int end)
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
COMPLETE_WITH("FULL", "FREEZE", "ANALYZE", "VERBOSE",
"DISABLE_PAGE_SKIPPING", "SKIP_LOCKED",
- "INDEX_CLEANUP", "PROCESS_TOAST",
+ "INDEX_CLEANUP", "PROCESS_MAIN", "PROCESS_TOAST",
"TRUNCATE", "PARALLEL");
- else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_TOAST|TRUNCATE"))
+ else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_MAIN|PROCESS_TOAST|TRUNCATE"))
COMPLETE_WITH("ON", "OFF");
else if (TailMatches("INDEX_CLEANUP"))
COMPLETE_WITH("AUTO", "ON", "OFF");
diff --git a/src/bin/scripts/t/100_vacuumdb.pl b/src/bin/scripts/t/100_vacuumdb.pl
index e5343774fe..fd7cb944cb 100644
--- a/src/bin/scripts/t/100_vacuumdb.pl
+++ b/src/bin/scripts/t/100_vacuumdb.pl
@@ -65,6 +65,13 @@ $node->issues_sql_like(
$node->command_fails(
[ 'vacuumdb', '--analyze-only', '--no-truncate', 'postgres' ],
'--analyze-only and --no-truncate specified together');
+$node->issues_sql_like(
+ [ 'vacuumdb', '--no-process-main', 'postgres' ],
+ qr/statement: VACUUM \(PROCESS_MAIN FALSE\).*;/,
+ 'vacuumdb --no-process-main');
+$node->command_fails(
+ [ 'vacuumdb', '--analyze-only', '--no-process-main', 'postgres' ],
+ '--analyze-only and --no-process_main specified together');
$node->issues_sql_like(
[ 'vacuumdb', '--no-process-toast', 'postgres' ],
qr/statement: VACUUM \(PROCESS_TOAST FALSE\).*;/,
diff --git a/src/bin/scripts/vacuumdb.c b/src/bin/scripts/vacuumdb.c
index 272e37d290..7424e75668 100644
--- a/src/bin/scripts/vacuumdb.c
+++ b/src/bin/scripts/vacuumdb.c
@@ -43,6 +43,7 @@ typedef struct vacuumingOptions
bool no_index_cleanup;
bool force_index_cleanup;
bool do_truncate;
+ bool process_main;
bool process_toast;
} vacuumingOptions;
@@ -120,6 +121,7 @@ main(int argc, char *argv[])
{"force-index-cleanup", no_argument, NULL, 9},
{"no-truncate", no_argument, NULL, 10},
{"no-process-toast", no_argument, NULL, 11},
+ {"no-process-main", no_argument, NULL, 12},
{NULL, 0, NULL, 0}
};
@@ -147,6 +149,7 @@ main(int argc, char *argv[])
vacopts.no_index_cleanup = false;
vacopts.force_index_cleanup = false;
vacopts.do_truncate = true;
+ vacopts.process_main = true;
vacopts.process_toast = true;
pg_logging_init(argv[0]);
@@ -259,6 +262,9 @@ main(int argc, char *argv[])
case 11:
vacopts.process_toast = false;
break;
+ case 12:
+ vacopts.process_main = false;
+ break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -311,6 +317,9 @@ main(int argc, char *argv[])
if (!vacopts.do_truncate)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-truncate");
+ if (!vacopts.process_main)
+ pg_fatal("cannot use the \"%s\" option when performing only analyze",
+ "no-process-main");
if (!vacopts.process_toast)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-process-toast");
@@ -507,6 +516,13 @@ vacuum_one_database(ConnParams *cparams,
"no-truncate", "12");
}
+ if (!vacopts->process_main && PQserverVersion(conn) < 160000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "no-process-main", "16");
+ }
+
if (!vacopts->process_toast && PQserverVersion(conn) < 140000)
{
PQfinish(conn);
@@ -950,6 +966,13 @@ prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
appendPQExpBuffer(sql, "%sTRUNCATE FALSE", sep);
sep = comma;
}
+ if (!vacopts->process_main)
+ {
+ /* PROCESS_MAIN is supported since v16 */
+ Assert(serverVersion >= 160000);
+ appendPQExpBuffer(sql, "%sPROCESS_MAIN FALSE", sep);
+ sep = comma;
+ }
if (!vacopts->process_toast)
{
/* PROCESS_TOAST is supported since v14 */
@@ -1057,6 +1080,7 @@ help(const char *progname)
printf(_(" --min-mxid-age=MXID_AGE minimum multixact ID age of tables to vacuum\n"));
printf(_(" --min-xid-age=XID_AGE minimum transaction ID age of tables to vacuum\n"));
printf(_(" --no-index-cleanup don't remove index entries that point to dead tuples\n"));
+ printf(_(" --no-process-main skip the main relation\n"));
printf(_(" --no-process-toast skip the TOAST table associated with the table to vacuum\n"));
printf(_(" --no-truncate don't truncate empty pages at the end of the table\n"));
printf(_(" -n, --schema=PATTERN vacuum tables in the specified schema(s) only\n"));
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 2f274f2bec..80fca14619 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -188,6 +188,7 @@ typedef struct VacAttrStats
#define VACOPT_SKIP_LOCKED 0x20 /* skip if cannot get lock */
#define VACOPT_PROCESS_TOAST 0x40 /* process the TOAST table, if any */
#define VACOPT_DISABLE_PAGE_SKIPPING 0x80 /* don't skip any pages */
+#define VACOPT_PROCESS_MAIN 0x100 /* process main relation */
/*
* Values used by index_cleanup and truncate params.
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index 0035d158b7..d3880f67b5 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -282,6 +282,10 @@ ALTER TABLE vactst ALTER COLUMN t SET STORAGE EXTERNAL;
VACUUM (PROCESS_TOAST FALSE) vactst;
VACUUM (PROCESS_TOAST FALSE, FULL) vactst;
ERROR: PROCESS_TOAST required with VACUUM FULL
+-- PROCESS_MAIN option
+VACUUM (PROCESS_MAIN FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 9faa8a34a6..d2893e0c37 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -237,6 +237,11 @@ ALTER TABLE vactst ALTER COLUMN t SET STORAGE EXTERNAL;
VACUUM (PROCESS_TOAST FALSE) vactst;
VACUUM (PROCESS_TOAST FALSE, FULL) vactst;
+-- PROCESS_MAIN option
+VACUUM (PROCESS_MAIN FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
--
2.25.1
rebased for cfbot
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Attachments:
v2-0001-add-PROCESS_MAIN-to-VACUUM.patchtext/x-diff; charset=us-asciiDownload
From 7e72b0a9f06fdfa00d5320d4c3303e67788878aa Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathandbossart@gmail.com>
Date: Thu, 29 Dec 2022 15:31:49 -0800
Subject: [PATCH v2 1/1] add PROCESS_MAIN to VACUUM
---
doc/src/sgml/ref/vacuum.sgml | 13 +++++++++++++
doc/src/sgml/ref/vacuumdb.sgml | 15 +++++++++++++++
src/backend/commands/vacuum.c | 27 +++++++++++++++++++++------
src/backend/postmaster/autovacuum.c | 4 +++-
src/bin/psql/tab-complete.c | 4 ++--
src/bin/scripts/t/100_vacuumdb.pl | 7 +++++++
src/bin/scripts/vacuumdb.c | 24 ++++++++++++++++++++++++
src/include/commands/vacuum.h | 1 +
src/test/regress/expected/vacuum.out | 4 ++++
src/test/regress/sql/vacuum.sql | 5 +++++
10 files changed, 95 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index 8fa8421847..4266f0c985 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -33,6 +33,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
DISABLE_PAGE_SKIPPING [ <replaceable class="parameter">boolean</replaceable> ]
SKIP_LOCKED [ <replaceable class="parameter">boolean</replaceable> ]
INDEX_CLEANUP { AUTO | ON | OFF }
+ PROCESS_MAIN [ <replaceable class="parameter">boolean</replaceable> ]
PROCESS_TOAST [ <replaceable class="parameter">boolean</replaceable> ]
TRUNCATE [ <replaceable class="parameter">boolean</replaceable> ]
PARALLEL <replaceable class="parameter">integer</replaceable>
@@ -238,6 +239,18 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>PROCESS_MAIN</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should attempt to process the
+ main relation. This is usually the desired behavior and is the default.
+ Setting this option to false may be useful when it is only necessary to
+ vacuum a relation's corresponding <literal>TOAST</literal> table.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>PROCESS_TOAST</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/vacuumdb.sgml b/doc/src/sgml/ref/vacuumdb.sgml
index 841aced3bd..74bac2d4ba 100644
--- a/doc/src/sgml/ref/vacuumdb.sgml
+++ b/doc/src/sgml/ref/vacuumdb.sgml
@@ -317,6 +317,21 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--no-process-main</option></term>
+ <listitem>
+ <para>
+ Skip the main relation.
+ </para>
+ <note>
+ <para>
+ This option is only available for servers running
+ <productname>PostgreSQL</productname> 16 and later.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--no-process-toast</option></term>
<listitem>
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index c4ed7efce3..3d8ae65a03 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -113,6 +113,7 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
bool freeze = false;
bool full = false;
bool disable_page_skipping = false;
+ bool process_main = true;
bool process_toast = true;
bool skip_database_stats = false;
bool only_database_stats = false;
@@ -166,6 +167,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
params.index_cleanup = get_vacoptval_from_boolean(opt);
}
}
+ else if (strcmp(opt->defname, "process_main") == 0)
+ process_main = defGetBoolean(opt);
else if (strcmp(opt->defname, "process_toast") == 0)
process_toast = defGetBoolean(opt);
else if (strcmp(opt->defname, "truncate") == 0)
@@ -224,7 +227,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
(disable_page_skipping ? VACOPT_DISABLE_PAGE_SKIPPING : 0) |
(process_toast ? VACOPT_PROCESS_TOAST : 0) |
(skip_database_stats ? VACOPT_SKIP_DATABASE_STATS : 0) |
- (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0);
+ (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0) |
+ (process_main ? VACOPT_PROCESS_MAIN : 0);
/* sanity checks on options */
Assert(params.options & (VACOPT_VACUUM | VACOPT_ANALYZE));
@@ -365,9 +369,10 @@ vacuum(List *relations, VacuumParams *params,
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ONLY_DATABASE_STATS cannot be specified with a list of tables")));
- /* don't require people to turn off PROCESS_TOAST explicitly */
+ /* don't require people to turn off PROCESS_TOAST/MAIN explicitly */
if (params->options & ~(VACOPT_VACUUM |
VACOPT_VERBOSE |
+ VACOPT_PROCESS_MAIN |
VACOPT_PROCESS_TOAST |
VACOPT_ONLY_DATABASE_STATS))
ereport(ERROR,
@@ -2026,10 +2031,12 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params)
/*
* Remember the relation's TOAST relation for later, if the caller asked
* us to process it. In VACUUM FULL, though, the toast table is
- * automatically rebuilt by cluster_rel so we shouldn't recurse to it.
+ * automatically rebuilt by cluster_rel so we shouldn't recurse to it
+ * unless PROCESS_MAIN is disabled.
*/
if ((params->options & VACOPT_PROCESS_TOAST) != 0 &&
- (params->options & VACOPT_FULL) == 0)
+ ((params->options & VACOPT_FULL) == 0 ||
+ (params->options & VACOPT_PROCESS_MAIN) == 0))
toast_relid = rel->rd_rel->reltoastrelid;
else
toast_relid = InvalidOid;
@@ -2048,7 +2055,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params)
/*
* Do the actual work --- either FULL or "lazy" vacuum
*/
- if (params->options & VACOPT_FULL)
+ if (params->options & VACOPT_FULL &&
+ params->options & VACOPT_PROCESS_MAIN)
{
ClusterParams cluster_params = {0};
@@ -2062,7 +2070,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params)
/* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
cluster_rel(relid, InvalidOid, &cluster_params);
}
- else
+ else if (params->options & VACOPT_PROCESS_MAIN)
table_relation_vacuum(rel, params, vac_strategy);
/* Roll back any GUC changes executed by index functions */
@@ -2089,7 +2097,14 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params)
* totally unimportant for toast relations.
*/
if (toast_relid != InvalidOid)
+ {
+ /* we force VACOPT_PROCESS_MAIN so vacuum_rel() processes it */
+ bool force_opt = ((params->options & VACOPT_PROCESS_MAIN) == 0);
+
+ params->options |= force_opt ? VACOPT_PROCESS_MAIN : 0;
vacuum_rel(toast_relid, NULL, params);
+ params->options &= force_opt ? ~VACOPT_PROCESS_MAIN : ~0;
+ }
/*
* Now release the session-level lock on the main table.
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index f5ea381c53..12dcb2b762 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -2860,7 +2860,9 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
* skip vac_update_datfrozenxid(); we'll do that separately.
*/
tab->at_params.options =
- (dovacuum ? (VACOPT_VACUUM | VACOPT_SKIP_DATABASE_STATS) : 0) |
+ (dovacuum ? (VACOPT_VACUUM |
+ VACOPT_PROCESS_MAIN |
+ VACOPT_SKIP_DATABASE_STATS) : 0) |
(doanalyze ? VACOPT_ANALYZE : 0) |
(!wraparound ? VACOPT_SKIP_LOCKED : 0);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 23750ea5fb..ef7bcc2a79 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -4597,10 +4597,10 @@ psql_completion(const char *text, int start, int end)
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
COMPLETE_WITH("FULL", "FREEZE", "ANALYZE", "VERBOSE",
"DISABLE_PAGE_SKIPPING", "SKIP_LOCKED",
- "INDEX_CLEANUP", "PROCESS_TOAST",
+ "INDEX_CLEANUP", "PROCESS_MAIN", "PROCESS_TOAST",
"TRUNCATE", "PARALLEL", "SKIP_DATABASE_STATS",
"ONLY_DATABASE_STATS");
- else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS"))
+ else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_MAIN|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS"))
COMPLETE_WITH("ON", "OFF");
else if (TailMatches("INDEX_CLEANUP"))
COMPLETE_WITH("AUTO", "ON", "OFF");
diff --git a/src/bin/scripts/t/100_vacuumdb.pl b/src/bin/scripts/t/100_vacuumdb.pl
index 3cfbaaec0d..46101899ae 100644
--- a/src/bin/scripts/t/100_vacuumdb.pl
+++ b/src/bin/scripts/t/100_vacuumdb.pl
@@ -65,6 +65,13 @@ $node->issues_sql_like(
$node->command_fails(
[ 'vacuumdb', '--analyze-only', '--no-truncate', 'postgres' ],
'--analyze-only and --no-truncate specified together');
+$node->issues_sql_like(
+ [ 'vacuumdb', '--no-process-main', 'postgres' ],
+ qr/statement: VACUUM \(PROCESS_MAIN FALSE, SKIP_DATABASE_STATS\).*;/,
+ 'vacuumdb --no-process-main');
+$node->command_fails(
+ [ 'vacuumdb', '--analyze-only', '--no-process-main', 'postgres' ],
+ '--analyze-only and --no-process_main specified together');
$node->issues_sql_like(
[ 'vacuumdb', '--no-process-toast', 'postgres' ],
qr/statement: VACUUM \(PROCESS_TOAST FALSE, SKIP_DATABASE_STATS\).*;/,
diff --git a/src/bin/scripts/vacuumdb.c b/src/bin/scripts/vacuumdb.c
index 58b894216b..39be265b5b 100644
--- a/src/bin/scripts/vacuumdb.c
+++ b/src/bin/scripts/vacuumdb.c
@@ -43,6 +43,7 @@ typedef struct vacuumingOptions
bool no_index_cleanup;
bool force_index_cleanup;
bool do_truncate;
+ bool process_main;
bool process_toast;
bool skip_database_stats;
} vacuumingOptions;
@@ -121,6 +122,7 @@ main(int argc, char *argv[])
{"force-index-cleanup", no_argument, NULL, 9},
{"no-truncate", no_argument, NULL, 10},
{"no-process-toast", no_argument, NULL, 11},
+ {"no-process-main", no_argument, NULL, 12},
{NULL, 0, NULL, 0}
};
@@ -148,6 +150,7 @@ main(int argc, char *argv[])
vacopts.no_index_cleanup = false;
vacopts.force_index_cleanup = false;
vacopts.do_truncate = true;
+ vacopts.process_main = true;
vacopts.process_toast = true;
pg_logging_init(argv[0]);
@@ -260,6 +263,9 @@ main(int argc, char *argv[])
case 11:
vacopts.process_toast = false;
break;
+ case 12:
+ vacopts.process_main = false;
+ break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -312,6 +318,9 @@ main(int argc, char *argv[])
if (!vacopts.do_truncate)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-truncate");
+ if (!vacopts.process_main)
+ pg_fatal("cannot use the \"%s\" option when performing only analyze",
+ "no-process-main");
if (!vacopts.process_toast)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-process-toast");
@@ -508,6 +517,13 @@ vacuum_one_database(ConnParams *cparams,
"no-truncate", "12");
}
+ if (!vacopts->process_main && PQserverVersion(conn) < 160000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "no-process-main", "16");
+ }
+
if (!vacopts->process_toast && PQserverVersion(conn) < 140000)
{
PQfinish(conn);
@@ -976,6 +992,13 @@ prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
appendPQExpBuffer(sql, "%sTRUNCATE FALSE", sep);
sep = comma;
}
+ if (!vacopts->process_main)
+ {
+ /* PROCESS_MAIN is supported since v16 */
+ Assert(serverVersion >= 160000);
+ appendPQExpBuffer(sql, "%sPROCESS_MAIN FALSE", sep);
+ sep = comma;
+ }
if (!vacopts->process_toast)
{
/* PROCESS_TOAST is supported since v14 */
@@ -1090,6 +1113,7 @@ help(const char *progname)
printf(_(" --min-mxid-age=MXID_AGE minimum multixact ID age of tables to vacuum\n"));
printf(_(" --min-xid-age=XID_AGE minimum transaction ID age of tables to vacuum\n"));
printf(_(" --no-index-cleanup don't remove index entries that point to dead tuples\n"));
+ printf(_(" --no-process-main skip the main relation\n"));
printf(_(" --no-process-toast skip the TOAST table associated with the table to vacuum\n"));
printf(_(" --no-truncate don't truncate empty pages at the end of the table\n"));
printf(_(" -n, --schema=PATTERN vacuum tables in the specified schema(s) only\n"));
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 689dbb7702..29f5c1af04 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -190,6 +190,7 @@ typedef struct VacAttrStats
#define VACOPT_DISABLE_PAGE_SKIPPING 0x80 /* don't skip any pages */
#define VACOPT_SKIP_DATABASE_STATS 0x100 /* skip vac_update_datfrozenxid() */
#define VACOPT_ONLY_DATABASE_STATS 0x200 /* only vac_update_datfrozenxid() */
+#define VACOPT_PROCESS_MAIN 0x400 /* process main relation */
/*
* Values used by index_cleanup and truncate params.
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index d860be0e20..e93594ad2a 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -288,6 +288,10 @@ VACUUM (SKIP_DATABASE_STATS) vactst;
VACUUM (ONLY_DATABASE_STATS);
VACUUM (ONLY_DATABASE_STATS) vactst; -- error
ERROR: ONLY_DATABASE_STATS cannot be specified with a list of tables
+-- PROCESS_MAIN option
+VACUUM (PROCESS_MAIN FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 9da8f3e830..aaa97aa8ba 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -244,6 +244,11 @@ VACUUM (SKIP_DATABASE_STATS) vactst;
VACUUM (ONLY_DATABASE_STATS);
VACUUM (ONLY_DATABASE_STATS) vactst; -- error
+-- PROCESS_MAIN option
+VACUUM (PROCESS_MAIN FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
--
2.25.1
On Thu, 2022-12-29 at 16:00 -0800, Nathan Bossart wrote:
The motivation for adding this option is to make it easier to VACUUM
only a
relation's TOAST table. At the moment, you need to find the TOAST
table by
examining a relation's reltoastrelid, and you need USAGE on the
pg_toast
schema. This option could also help make it possible to call only
vac_update_datfrozenxid() without processing any relations, as
discussed
elsewhere [2].
For completeness, did you consider CLUSTER and REINDEX options as well?
--
Jeff Davis
PostgreSQL Contributor Team - AWS
On Fri, Jan 13, 2023 at 03:24:09PM -0800, Jeff Davis wrote:
For completeness, did you consider CLUSTER and REINDEX options as well?
I have not, but I can put together patches for those as well.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
On Sat, 7 Jan 2023 at 10:37, Nathan Bossart <nathandbossart@gmail.com> wrote:
rebased for cfbot
The patch does not apply on top of HEAD as in [1]http://cfbot.cputube.org/patch_41_4088.log, please post a rebased patch:
=== Applying patches on top of PostgreSQL commit ID
d540a02a724b9643205abce8c5644a0f0908f6e3 ===
=== applying patch ./v2-0001-add-PROCESS_MAIN-to-VACUUM.patch
patching file src/backend/commands/vacuum.c
....
Hunk #8 FAILED at 2097.
1 out of 8 hunks FAILED -- saving rejects to file
src/backend/commands/vacuum.c.rej
[1]: http://cfbot.cputube.org/patch_41_4088.log
Regards,
Vignesh
On Thu, Jan 19, 2023 at 05:28:25PM +0530, vignesh C wrote:
The patch does not apply on top of HEAD as in [1], please post a rebased patch:
rebased
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Attachments:
v3-0001-add-PROCESS_MAIN-to-VACUUM.patchtext/x-diff; charset=us-asciiDownload
From 7a7f96bf4eea5be6cc252dda6bc330e77a6a3316 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathandbossart@gmail.com>
Date: Thu, 29 Dec 2022 15:31:49 -0800
Subject: [PATCH v3 1/1] add PROCESS_MAIN to VACUUM
---
doc/src/sgml/ref/vacuum.sgml | 13 +++++++++++++
doc/src/sgml/ref/vacuumdb.sgml | 15 +++++++++++++++
src/backend/commands/vacuum.c | 28 ++++++++++++++++++++++------
src/backend/postmaster/autovacuum.c | 4 +++-
src/bin/psql/tab-complete.c | 4 ++--
src/bin/scripts/t/100_vacuumdb.pl | 7 +++++++
src/bin/scripts/vacuumdb.c | 24 ++++++++++++++++++++++++
src/include/commands/vacuum.h | 1 +
src/test/regress/expected/vacuum.out | 4 ++++
src/test/regress/sql/vacuum.sql | 5 +++++
10 files changed, 96 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index 545b23b54f..b6d30b5764 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -33,6 +33,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
DISABLE_PAGE_SKIPPING [ <replaceable class="parameter">boolean</replaceable> ]
SKIP_LOCKED [ <replaceable class="parameter">boolean</replaceable> ]
INDEX_CLEANUP { AUTO | ON | OFF }
+ PROCESS_MAIN [ <replaceable class="parameter">boolean</replaceable> ]
PROCESS_TOAST [ <replaceable class="parameter">boolean</replaceable> ]
TRUNCATE [ <replaceable class="parameter">boolean</replaceable> ]
PARALLEL <replaceable class="parameter">integer</replaceable>
@@ -238,6 +239,18 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>PROCESS_MAIN</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should attempt to process the
+ main relation. This is usually the desired behavior and is the default.
+ Setting this option to false may be useful when it is only necessary to
+ vacuum a relation's corresponding <literal>TOAST</literal> table.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>PROCESS_TOAST</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/vacuumdb.sgml b/doc/src/sgml/ref/vacuumdb.sgml
index 841aced3bd..74bac2d4ba 100644
--- a/doc/src/sgml/ref/vacuumdb.sgml
+++ b/doc/src/sgml/ref/vacuumdb.sgml
@@ -317,6 +317,21 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--no-process-main</option></term>
+ <listitem>
+ <para>
+ Skip the main relation.
+ </para>
+ <note>
+ <para>
+ This option is only available for servers running
+ <productname>PostgreSQL</productname> 16 and later.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--no-process-toast</option></term>
<listitem>
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 7b1a4b127e..eea39317e9 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -115,6 +115,7 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
bool freeze = false;
bool full = false;
bool disable_page_skipping = false;
+ bool process_main = true;
bool process_toast = true;
bool skip_database_stats = false;
bool only_database_stats = false;
@@ -168,6 +169,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
params.index_cleanup = get_vacoptval_from_boolean(opt);
}
}
+ else if (strcmp(opt->defname, "process_main") == 0)
+ process_main = defGetBoolean(opt);
else if (strcmp(opt->defname, "process_toast") == 0)
process_toast = defGetBoolean(opt);
else if (strcmp(opt->defname, "truncate") == 0)
@@ -226,7 +229,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
(disable_page_skipping ? VACOPT_DISABLE_PAGE_SKIPPING : 0) |
(process_toast ? VACOPT_PROCESS_TOAST : 0) |
(skip_database_stats ? VACOPT_SKIP_DATABASE_STATS : 0) |
- (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0);
+ (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0) |
+ (process_main ? VACOPT_PROCESS_MAIN : 0);
/* sanity checks on options */
Assert(params.options & (VACOPT_VACUUM | VACOPT_ANALYZE));
@@ -367,9 +371,10 @@ vacuum(List *relations, VacuumParams *params,
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ONLY_DATABASE_STATS cannot be specified with a list of tables")));
- /* don't require people to turn off PROCESS_TOAST explicitly */
+ /* don't require people to turn off PROCESS_TOAST/MAIN explicitly */
if (params->options & ~(VACOPT_VACUUM |
VACOPT_VERBOSE |
+ VACOPT_PROCESS_MAIN |
VACOPT_PROCESS_TOAST |
VACOPT_ONLY_DATABASE_STATS))
ereport(ERROR,
@@ -2031,10 +2036,12 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/*
* Remember the relation's TOAST relation for later, if the caller asked
* us to process it. In VACUUM FULL, though, the toast table is
- * automatically rebuilt by cluster_rel so we shouldn't recurse to it.
+ * automatically rebuilt by cluster_rel so we shouldn't recurse to it
+ * unless PROCESS_MAIN is disabled.
*/
if ((params->options & VACOPT_PROCESS_TOAST) != 0 &&
- (params->options & VACOPT_FULL) == 0)
+ ((params->options & VACOPT_FULL) == 0 ||
+ (params->options & VACOPT_PROCESS_MAIN) == 0))
toast_relid = rel->rd_rel->reltoastrelid;
else
toast_relid = InvalidOid;
@@ -2053,7 +2060,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/*
* Do the actual work --- either FULL or "lazy" vacuum
*/
- if (params->options & VACOPT_FULL)
+ if (params->options & VACOPT_FULL &&
+ params->options & VACOPT_PROCESS_MAIN)
{
ClusterParams cluster_params = {0};
@@ -2067,7 +2075,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
cluster_rel(relid, InvalidOid, &cluster_params);
}
- else
+ else if (params->options & VACOPT_PROCESS_MAIN)
table_relation_vacuum(rel, params, vac_strategy);
/* Roll back any GUC changes executed by index functions */
@@ -2094,7 +2102,15 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
* totally unimportant for toast relations.
*/
if (toast_relid != InvalidOid)
+ {
+ /* we force VACOPT_PROCESS_MAIN so vacuum_rel() processes it */
+ bool force_opt = ((params->options & VACOPT_PROCESS_MAIN) == 0);
+
+ params->options |= VACOPT_PROCESS_MAIN;
vacuum_rel(toast_relid, NULL, params, true);
+ if (force_opt)
+ params->options &= ~VACOPT_PROCESS_MAIN;
+ }
/*
* Now release the session-level lock on the main table.
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index f5ea381c53..12dcb2b762 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -2860,7 +2860,9 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
* skip vac_update_datfrozenxid(); we'll do that separately.
*/
tab->at_params.options =
- (dovacuum ? (VACOPT_VACUUM | VACOPT_SKIP_DATABASE_STATS) : 0) |
+ (dovacuum ? (VACOPT_VACUUM |
+ VACOPT_PROCESS_MAIN |
+ VACOPT_SKIP_DATABASE_STATS) : 0) |
(doanalyze ? VACOPT_ANALYZE : 0) |
(!wraparound ? VACOPT_SKIP_LOCKED : 0);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5e1882eaea..8f12af799b 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -4618,10 +4618,10 @@ psql_completion(const char *text, int start, int end)
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
COMPLETE_WITH("FULL", "FREEZE", "ANALYZE", "VERBOSE",
"DISABLE_PAGE_SKIPPING", "SKIP_LOCKED",
- "INDEX_CLEANUP", "PROCESS_TOAST",
+ "INDEX_CLEANUP", "PROCESS_MAIN", "PROCESS_TOAST",
"TRUNCATE", "PARALLEL", "SKIP_DATABASE_STATS",
"ONLY_DATABASE_STATS");
- else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS"))
+ else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_MAIN|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS"))
COMPLETE_WITH("ON", "OFF");
else if (TailMatches("INDEX_CLEANUP"))
COMPLETE_WITH("AUTO", "ON", "OFF");
diff --git a/src/bin/scripts/t/100_vacuumdb.pl b/src/bin/scripts/t/100_vacuumdb.pl
index 3cfbaaec0d..46101899ae 100644
--- a/src/bin/scripts/t/100_vacuumdb.pl
+++ b/src/bin/scripts/t/100_vacuumdb.pl
@@ -65,6 +65,13 @@ $node->issues_sql_like(
$node->command_fails(
[ 'vacuumdb', '--analyze-only', '--no-truncate', 'postgres' ],
'--analyze-only and --no-truncate specified together');
+$node->issues_sql_like(
+ [ 'vacuumdb', '--no-process-main', 'postgres' ],
+ qr/statement: VACUUM \(PROCESS_MAIN FALSE, SKIP_DATABASE_STATS\).*;/,
+ 'vacuumdb --no-process-main');
+$node->command_fails(
+ [ 'vacuumdb', '--analyze-only', '--no-process-main', 'postgres' ],
+ '--analyze-only and --no-process_main specified together');
$node->issues_sql_like(
[ 'vacuumdb', '--no-process-toast', 'postgres' ],
qr/statement: VACUUM \(PROCESS_TOAST FALSE, SKIP_DATABASE_STATS\).*;/,
diff --git a/src/bin/scripts/vacuumdb.c b/src/bin/scripts/vacuumdb.c
index 58b894216b..39be265b5b 100644
--- a/src/bin/scripts/vacuumdb.c
+++ b/src/bin/scripts/vacuumdb.c
@@ -43,6 +43,7 @@ typedef struct vacuumingOptions
bool no_index_cleanup;
bool force_index_cleanup;
bool do_truncate;
+ bool process_main;
bool process_toast;
bool skip_database_stats;
} vacuumingOptions;
@@ -121,6 +122,7 @@ main(int argc, char *argv[])
{"force-index-cleanup", no_argument, NULL, 9},
{"no-truncate", no_argument, NULL, 10},
{"no-process-toast", no_argument, NULL, 11},
+ {"no-process-main", no_argument, NULL, 12},
{NULL, 0, NULL, 0}
};
@@ -148,6 +150,7 @@ main(int argc, char *argv[])
vacopts.no_index_cleanup = false;
vacopts.force_index_cleanup = false;
vacopts.do_truncate = true;
+ vacopts.process_main = true;
vacopts.process_toast = true;
pg_logging_init(argv[0]);
@@ -260,6 +263,9 @@ main(int argc, char *argv[])
case 11:
vacopts.process_toast = false;
break;
+ case 12:
+ vacopts.process_main = false;
+ break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -312,6 +318,9 @@ main(int argc, char *argv[])
if (!vacopts.do_truncate)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-truncate");
+ if (!vacopts.process_main)
+ pg_fatal("cannot use the \"%s\" option when performing only analyze",
+ "no-process-main");
if (!vacopts.process_toast)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-process-toast");
@@ -508,6 +517,13 @@ vacuum_one_database(ConnParams *cparams,
"no-truncate", "12");
}
+ if (!vacopts->process_main && PQserverVersion(conn) < 160000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "no-process-main", "16");
+ }
+
if (!vacopts->process_toast && PQserverVersion(conn) < 140000)
{
PQfinish(conn);
@@ -976,6 +992,13 @@ prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
appendPQExpBuffer(sql, "%sTRUNCATE FALSE", sep);
sep = comma;
}
+ if (!vacopts->process_main)
+ {
+ /* PROCESS_MAIN is supported since v16 */
+ Assert(serverVersion >= 160000);
+ appendPQExpBuffer(sql, "%sPROCESS_MAIN FALSE", sep);
+ sep = comma;
+ }
if (!vacopts->process_toast)
{
/* PROCESS_TOAST is supported since v14 */
@@ -1090,6 +1113,7 @@ help(const char *progname)
printf(_(" --min-mxid-age=MXID_AGE minimum multixact ID age of tables to vacuum\n"));
printf(_(" --min-xid-age=XID_AGE minimum transaction ID age of tables to vacuum\n"));
printf(_(" --no-index-cleanup don't remove index entries that point to dead tuples\n"));
+ printf(_(" --no-process-main skip the main relation\n"));
printf(_(" --no-process-toast skip the TOAST table associated with the table to vacuum\n"));
printf(_(" --no-truncate don't truncate empty pages at the end of the table\n"));
printf(_(" -n, --schema=PATTERN vacuum tables in the specified schema(s) only\n"));
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 689dbb7702..29f5c1af04 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -190,6 +190,7 @@ typedef struct VacAttrStats
#define VACOPT_DISABLE_PAGE_SKIPPING 0x80 /* don't skip any pages */
#define VACOPT_SKIP_DATABASE_STATS 0x100 /* skip vac_update_datfrozenxid() */
#define VACOPT_ONLY_DATABASE_STATS 0x200 /* only vac_update_datfrozenxid() */
+#define VACOPT_PROCESS_MAIN 0x400 /* process main relation */
/*
* Values used by index_cleanup and truncate params.
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index 458adee7f8..e482873436 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -288,6 +288,10 @@ VACUUM (SKIP_DATABASE_STATS) vactst;
VACUUM (ONLY_DATABASE_STATS);
VACUUM (ONLY_DATABASE_STATS) vactst; -- error
ERROR: ONLY_DATABASE_STATS cannot be specified with a list of tables
+-- PROCESS_MAIN option
+VACUUM (PROCESS_MAIN FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 9da8f3e830..aaa97aa8ba 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -244,6 +244,11 @@ VACUUM (SKIP_DATABASE_STATS) vactst;
VACUUM (ONLY_DATABASE_STATS);
VACUUM (ONLY_DATABASE_STATS) vactst; -- error
+-- PROCESS_MAIN option
+VACUUM (PROCESS_MAIN FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
+
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
--
2.25.1
On Fri, Jan 13, 2023 at 03:30:15PM -0800, Nathan Bossart wrote:
On Fri, Jan 13, 2023 at 03:24:09PM -0800, Jeff Davis wrote:
For completeness, did you consider CLUSTER and REINDEX options as well?
I have not, but I can put together patches for those as well.
Are you planning to do that here, on this thread ?
It seems like a good idea - it would allow simplifying an existing or
future scripts which needs to reindex a toast table, saving the trouble
of looking up the name of the toast table, and a race condition and
server error log if the table itself were processed. I can imagine that
might happen if a separate process used TRUNCATE, for example.
--
Justin
On Mon, Feb 20, 2023 at 10:31:11AM -0600, Justin Pryzby wrote:
On Fri, Jan 13, 2023 at 03:30:15PM -0800, Nathan Bossart wrote:
On Fri, Jan 13, 2023 at 03:24:09PM -0800, Jeff Davis wrote:
For completeness, did you consider CLUSTER and REINDEX options as well?
I have not, but I can put together patches for those as well.
Are you planning to do that here, on this thread ?
Yes, I just haven't made time for it yet. IIRC I briefly looked into
CLUSTER and decided that it was probably not worth the effort, but I still
think it's worth adding this option to REINDEX.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
On Thu, Jan 19, 2023 at 11:08:07AM -0800, Nathan Bossart wrote:
rebased
PROCESS_TOAST has that:
/* sanity check for PROCESS_TOAST */
if ((params->options & VACOPT_FULL) != 0 &&
(params->options & VACOPT_PROCESS_TOAST) == 0)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("PROCESS_TOAST required with VACUUM FULL")));
[...]
- if (params->options & VACOPT_FULL)
+ if (params->options & VACOPT_FULL &&
+ params->options & VACOPT_PROCESS_MAIN)
{
Shouldn't we apply the same rule for PROCESS_MAIN? One of the
regression tests added means that FULL takes priority over
PROCESS_MAIN=FALSE, which is a bit confusing IMO.
@@ -190,6 +190,7 @@ typedef struct VacAttrStats
#define VACOPT_DISABLE_PAGE_SKIPPING 0x80 /* don't skip any pages */
#define VACOPT_SKIP_DATABASE_STATS 0x100 /* skip vac_update_datfrozenxid() */
#define VACOPT_ONLY_DATABASE_STATS 0x200 /* only vac_update_datfrozenxid() */
+#define VACOPT_PROCESS_MAIN 0x400 /* process main relation */
Perhaps the options had better be reorganized so as PROCESS_MAIN is
just before PROCESS_TOAST?
+-- PROCESS_MAIN option
+VACUUM (PROCESS_MAIN FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
Thinking a bit here. This set of tests does not make sure that the
main relation and/or the toast relation have been actually processed.
pg_stat_user_tables does not track what's happening on the toast
relations. So... What about adding some tests in 100_vacuumdb.pl
that rely on vacuumdb --verbose and check the logs produced? We
should make sure that the toast or the main relation are processed,
by tracking, for example, logs like vacuuming "schema.table". When
FULL is involved, we may want to track the changes on relfilenodes
depending on what's wanted.
--
Michael
Thanks for taking a look.
On Wed, Mar 01, 2023 at 03:31:48PM +0900, Michael Paquier wrote:
PROCESS_TOAST has that: /* sanity check for PROCESS_TOAST */ if ((params->options & VACOPT_FULL) != 0 && (params->options & VACOPT_PROCESS_TOAST) == 0) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("PROCESS_TOAST required with VACUUM FULL"))); [...] - if (params->options & VACOPT_FULL) + if (params->options & VACOPT_FULL && + params->options & VACOPT_PROCESS_MAIN) {Shouldn't we apply the same rule for PROCESS_MAIN? One of the
regression tests added means that FULL takes priority over
PROCESS_MAIN=FALSE, which is a bit confusing IMO.
I don't think so. We disallow FULL without PROCESS_TOAST because there
presently isn't a way to VACUUM FULL the main relation without rebuilding
its TOAST table. However, FULL without PROCESS_MAIN can be used to run
VACUUM FULL on only the TOAST table.
@@ -190,6 +190,7 @@ typedef struct VacAttrStats #define VACOPT_DISABLE_PAGE_SKIPPING 0x80 /* don't skip any pages */ #define VACOPT_SKIP_DATABASE_STATS 0x100 /* skip vac_update_datfrozenxid() */ #define VACOPT_ONLY_DATABASE_STATS 0x200 /* only vac_update_datfrozenxid() */ +#define VACOPT_PROCESS_MAIN 0x400 /* process main relation */Perhaps the options had better be reorganized so as PROCESS_MAIN is
just before PROCESS_TOAST?
Sure.
+-- PROCESS_MAIN option +VACUUM (PROCESS_MAIN FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, FULL) vactst;Thinking a bit here. This set of tests does not make sure that the
main relation and/or the toast relation have been actually processed.
pg_stat_user_tables does not track what's happening on the toast
relations. So... What about adding some tests in 100_vacuumdb.pl
that rely on vacuumdb --verbose and check the logs produced? We
should make sure that the toast or the main relation are processed,
by tracking, for example, logs like vacuuming "schema.table". When
FULL is involved, we may want to track the changes on relfilenodes
depending on what's wanted.
That seems like a good idea.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
On 2023-Mar-01, Michael Paquier wrote:
+-- PROCESS_MAIN option +VACUUM (PROCESS_MAIN FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, FULL) vactst;Thinking a bit here. This set of tests does not make sure that the
main relation and/or the toast relation have been actually processed.
pg_stat_user_tables does not track what's happening on the toast
relations. So... What about adding some tests in 100_vacuumdb.pl
that rely on vacuumdb --verbose and check the logs produced? We
should make sure that the toast or the main relation are processed,
by tracking, for example, logs like vacuuming "schema.table". When
FULL is involved, we may want to track the changes on relfilenodes
depending on what's wanted.
Maybe instead of reading the log, read values from pg_stat_all_tables.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Uno puede defenderse de los ataques; contra los elogios se esta indefenso"
On Wed, Mar 01, 2023 at 07:09:53PM +0100, Alvaro Herrera wrote:
On 2023-Mar-01, Michael Paquier wrote:
+-- PROCESS_MAIN option +VACUUM (PROCESS_MAIN FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, FULL) vactst;Thinking a bit here. This set of tests does not make sure that the
main relation and/or the toast relation have been actually processed.
pg_stat_user_tables does not track what's happening on the toast
relations. So... What about adding some tests in 100_vacuumdb.pl
that rely on vacuumdb --verbose and check the logs produced? We
should make sure that the toast or the main relation are processed,
by tracking, for example, logs like vacuuming "schema.table". When
FULL is involved, we may want to track the changes on relfilenodes
depending on what's wanted.Maybe instead of reading the log, read values from pg_stat_all_tables.
Here is an attempt at that. Thanks for the idea.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Attachments:
v4-0001-add-PROCESS_MAIN-to-VACUUM.patchtext/x-diff; charset=us-asciiDownload
From 3b628a96fdebf1a052472de307f36b20a5a1414b Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathandbossart@gmail.com>
Date: Thu, 29 Dec 2022 15:31:49 -0800
Subject: [PATCH v4 1/1] add PROCESS_MAIN to VACUUM
---
doc/src/sgml/ref/vacuum.sgml | 13 ++++++
doc/src/sgml/ref/vacuumdb.sgml | 15 +++++++
src/backend/commands/vacuum.c | 28 ++++++++++---
src/backend/postmaster/autovacuum.c | 4 +-
src/bin/psql/tab-complete.c | 4 +-
src/bin/scripts/t/100_vacuumdb.pl | 7 ++++
src/bin/scripts/vacuumdb.c | 24 +++++++++++
src/include/commands/vacuum.h | 9 +++--
src/test/regress/expected/vacuum.out | 60 ++++++++++++++++++++++++++++
src/test/regress/sql/vacuum.sql | 24 +++++++++++
10 files changed, 175 insertions(+), 13 deletions(-)
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index 545b23b54f..b6d30b5764 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -33,6 +33,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
DISABLE_PAGE_SKIPPING [ <replaceable class="parameter">boolean</replaceable> ]
SKIP_LOCKED [ <replaceable class="parameter">boolean</replaceable> ]
INDEX_CLEANUP { AUTO | ON | OFF }
+ PROCESS_MAIN [ <replaceable class="parameter">boolean</replaceable> ]
PROCESS_TOAST [ <replaceable class="parameter">boolean</replaceable> ]
TRUNCATE [ <replaceable class="parameter">boolean</replaceable> ]
PARALLEL <replaceable class="parameter">integer</replaceable>
@@ -238,6 +239,18 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>PROCESS_MAIN</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should attempt to process the
+ main relation. This is usually the desired behavior and is the default.
+ Setting this option to false may be useful when it is only necessary to
+ vacuum a relation's corresponding <literal>TOAST</literal> table.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>PROCESS_TOAST</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/vacuumdb.sgml b/doc/src/sgml/ref/vacuumdb.sgml
index 841aced3bd..74bac2d4ba 100644
--- a/doc/src/sgml/ref/vacuumdb.sgml
+++ b/doc/src/sgml/ref/vacuumdb.sgml
@@ -317,6 +317,21 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--no-process-main</option></term>
+ <listitem>
+ <para>
+ Skip the main relation.
+ </para>
+ <note>
+ <para>
+ This option is only available for servers running
+ <productname>PostgreSQL</productname> 16 and later.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--no-process-toast</option></term>
<listitem>
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index aa79d9de4d..88ac1df2b5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -115,6 +115,7 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
bool freeze = false;
bool full = false;
bool disable_page_skipping = false;
+ bool process_main = true;
bool process_toast = true;
bool skip_database_stats = false;
bool only_database_stats = false;
@@ -168,6 +169,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
params.index_cleanup = get_vacoptval_from_boolean(opt);
}
}
+ else if (strcmp(opt->defname, "process_main") == 0)
+ process_main = defGetBoolean(opt);
else if (strcmp(opt->defname, "process_toast") == 0)
process_toast = defGetBoolean(opt);
else if (strcmp(opt->defname, "truncate") == 0)
@@ -226,7 +229,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
(disable_page_skipping ? VACOPT_DISABLE_PAGE_SKIPPING : 0) |
(process_toast ? VACOPT_PROCESS_TOAST : 0) |
(skip_database_stats ? VACOPT_SKIP_DATABASE_STATS : 0) |
- (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0);
+ (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0) |
+ (process_main ? VACOPT_PROCESS_MAIN : 0);
/* sanity checks on options */
Assert(params.options & (VACOPT_VACUUM | VACOPT_ANALYZE));
@@ -367,9 +371,10 @@ vacuum(List *relations, VacuumParams *params,
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ONLY_DATABASE_STATS cannot be specified with a list of tables")));
- /* don't require people to turn off PROCESS_TOAST explicitly */
+ /* don't require people to turn off PROCESS_TOAST/MAIN explicitly */
if (params->options & ~(VACOPT_VACUUM |
VACOPT_VERBOSE |
+ VACOPT_PROCESS_MAIN |
VACOPT_PROCESS_TOAST |
VACOPT_ONLY_DATABASE_STATS))
ereport(ERROR,
@@ -2031,10 +2036,12 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/*
* Remember the relation's TOAST relation for later, if the caller asked
* us to process it. In VACUUM FULL, though, the toast table is
- * automatically rebuilt by cluster_rel so we shouldn't recurse to it.
+ * automatically rebuilt by cluster_rel so we shouldn't recurse to it
+ * unless PROCESS_MAIN is disabled.
*/
if ((params->options & VACOPT_PROCESS_TOAST) != 0 &&
- (params->options & VACOPT_FULL) == 0)
+ ((params->options & VACOPT_FULL) == 0 ||
+ (params->options & VACOPT_PROCESS_MAIN) == 0))
toast_relid = rel->rd_rel->reltoastrelid;
else
toast_relid = InvalidOid;
@@ -2053,7 +2060,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/*
* Do the actual work --- either FULL or "lazy" vacuum
*/
- if (params->options & VACOPT_FULL)
+ if (params->options & VACOPT_FULL &&
+ params->options & VACOPT_PROCESS_MAIN)
{
ClusterParams cluster_params = {0};
@@ -2067,7 +2075,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
cluster_rel(relid, InvalidOid, &cluster_params);
}
- else
+ else if (params->options & VACOPT_PROCESS_MAIN)
table_relation_vacuum(rel, params, vac_strategy);
/* Roll back any GUC changes executed by index functions */
@@ -2094,7 +2102,15 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
* totally unimportant for toast relations.
*/
if (toast_relid != InvalidOid)
+ {
+ /* we force VACOPT_PROCESS_MAIN so vacuum_rel() processes it */
+ bool force_opt = ((params->options & VACOPT_PROCESS_MAIN) == 0);
+
+ params->options |= VACOPT_PROCESS_MAIN;
vacuum_rel(toast_relid, NULL, params, true);
+ if (force_opt)
+ params->options &= ~VACOPT_PROCESS_MAIN;
+ }
/*
* Now release the session-level lock on the main table.
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index ff6149a179..c0e2e00a7e 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -2860,7 +2860,9 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
* skip vac_update_datfrozenxid(); we'll do that separately.
*/
tab->at_params.options =
- (dovacuum ? (VACOPT_VACUUM | VACOPT_SKIP_DATABASE_STATS) : 0) |
+ (dovacuum ? (VACOPT_VACUUM |
+ VACOPT_PROCESS_MAIN |
+ VACOPT_SKIP_DATABASE_STATS) : 0) |
(doanalyze ? VACOPT_ANALYZE : 0) |
(!wraparound ? VACOPT_SKIP_LOCKED : 0);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5e1882eaea..8f12af799b 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -4618,10 +4618,10 @@ psql_completion(const char *text, int start, int end)
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
COMPLETE_WITH("FULL", "FREEZE", "ANALYZE", "VERBOSE",
"DISABLE_PAGE_SKIPPING", "SKIP_LOCKED",
- "INDEX_CLEANUP", "PROCESS_TOAST",
+ "INDEX_CLEANUP", "PROCESS_MAIN", "PROCESS_TOAST",
"TRUNCATE", "PARALLEL", "SKIP_DATABASE_STATS",
"ONLY_DATABASE_STATS");
- else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS"))
+ else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_MAIN|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS"))
COMPLETE_WITH("ON", "OFF");
else if (TailMatches("INDEX_CLEANUP"))
COMPLETE_WITH("AUTO", "ON", "OFF");
diff --git a/src/bin/scripts/t/100_vacuumdb.pl b/src/bin/scripts/t/100_vacuumdb.pl
index 3cfbaaec0d..46101899ae 100644
--- a/src/bin/scripts/t/100_vacuumdb.pl
+++ b/src/bin/scripts/t/100_vacuumdb.pl
@@ -65,6 +65,13 @@ $node->issues_sql_like(
$node->command_fails(
[ 'vacuumdb', '--analyze-only', '--no-truncate', 'postgres' ],
'--analyze-only and --no-truncate specified together');
+$node->issues_sql_like(
+ [ 'vacuumdb', '--no-process-main', 'postgres' ],
+ qr/statement: VACUUM \(PROCESS_MAIN FALSE, SKIP_DATABASE_STATS\).*;/,
+ 'vacuumdb --no-process-main');
+$node->command_fails(
+ [ 'vacuumdb', '--analyze-only', '--no-process-main', 'postgres' ],
+ '--analyze-only and --no-process_main specified together');
$node->issues_sql_like(
[ 'vacuumdb', '--no-process-toast', 'postgres' ],
qr/statement: VACUUM \(PROCESS_TOAST FALSE, SKIP_DATABASE_STATS\).*;/,
diff --git a/src/bin/scripts/vacuumdb.c b/src/bin/scripts/vacuumdb.c
index 58b894216b..39be265b5b 100644
--- a/src/bin/scripts/vacuumdb.c
+++ b/src/bin/scripts/vacuumdb.c
@@ -43,6 +43,7 @@ typedef struct vacuumingOptions
bool no_index_cleanup;
bool force_index_cleanup;
bool do_truncate;
+ bool process_main;
bool process_toast;
bool skip_database_stats;
} vacuumingOptions;
@@ -121,6 +122,7 @@ main(int argc, char *argv[])
{"force-index-cleanup", no_argument, NULL, 9},
{"no-truncate", no_argument, NULL, 10},
{"no-process-toast", no_argument, NULL, 11},
+ {"no-process-main", no_argument, NULL, 12},
{NULL, 0, NULL, 0}
};
@@ -148,6 +150,7 @@ main(int argc, char *argv[])
vacopts.no_index_cleanup = false;
vacopts.force_index_cleanup = false;
vacopts.do_truncate = true;
+ vacopts.process_main = true;
vacopts.process_toast = true;
pg_logging_init(argv[0]);
@@ -260,6 +263,9 @@ main(int argc, char *argv[])
case 11:
vacopts.process_toast = false;
break;
+ case 12:
+ vacopts.process_main = false;
+ break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -312,6 +318,9 @@ main(int argc, char *argv[])
if (!vacopts.do_truncate)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-truncate");
+ if (!vacopts.process_main)
+ pg_fatal("cannot use the \"%s\" option when performing only analyze",
+ "no-process-main");
if (!vacopts.process_toast)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-process-toast");
@@ -508,6 +517,13 @@ vacuum_one_database(ConnParams *cparams,
"no-truncate", "12");
}
+ if (!vacopts->process_main && PQserverVersion(conn) < 160000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "no-process-main", "16");
+ }
+
if (!vacopts->process_toast && PQserverVersion(conn) < 140000)
{
PQfinish(conn);
@@ -976,6 +992,13 @@ prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
appendPQExpBuffer(sql, "%sTRUNCATE FALSE", sep);
sep = comma;
}
+ if (!vacopts->process_main)
+ {
+ /* PROCESS_MAIN is supported since v16 */
+ Assert(serverVersion >= 160000);
+ appendPQExpBuffer(sql, "%sPROCESS_MAIN FALSE", sep);
+ sep = comma;
+ }
if (!vacopts->process_toast)
{
/* PROCESS_TOAST is supported since v14 */
@@ -1090,6 +1113,7 @@ help(const char *progname)
printf(_(" --min-mxid-age=MXID_AGE minimum multixact ID age of tables to vacuum\n"));
printf(_(" --min-xid-age=XID_AGE minimum transaction ID age of tables to vacuum\n"));
printf(_(" --no-index-cleanup don't remove index entries that point to dead tuples\n"));
+ printf(_(" --no-process-main skip the main relation\n"));
printf(_(" --no-process-toast skip the TOAST table associated with the table to vacuum\n"));
printf(_(" --no-truncate don't truncate empty pages at the end of the table\n"));
printf(_(" -n, --schema=PATTERN vacuum tables in the specified schema(s) only\n"));
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 689dbb7702..bdfd96cfec 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -186,10 +186,11 @@ typedef struct VacAttrStats
#define VACOPT_FREEZE 0x08 /* FREEZE option */
#define VACOPT_FULL 0x10 /* FULL (non-concurrent) vacuum */
#define VACOPT_SKIP_LOCKED 0x20 /* skip if cannot get lock */
-#define VACOPT_PROCESS_TOAST 0x40 /* process the TOAST table, if any */
-#define VACOPT_DISABLE_PAGE_SKIPPING 0x80 /* don't skip any pages */
-#define VACOPT_SKIP_DATABASE_STATS 0x100 /* skip vac_update_datfrozenxid() */
-#define VACOPT_ONLY_DATABASE_STATS 0x200 /* only vac_update_datfrozenxid() */
+#define VACOPT_PROCESS_MAIN 0x40 /* process main relation */
+#define VACOPT_PROCESS_TOAST 0x80 /* process the TOAST table, if any */
+#define VACOPT_DISABLE_PAGE_SKIPPING 0x100 /* don't skip any pages */
+#define VACOPT_SKIP_DATABASE_STATS 0x200 /* skip vac_update_datfrozenxid() */
+#define VACOPT_ONLY_DATABASE_STATS 0x400 /* only vac_update_datfrozenxid() */
/*
* Values used by index_cleanup and truncate params.
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index 458adee7f8..17fba0fe06 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -276,10 +276,29 @@ RESET default_transaction_isolation;
BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
ANALYZE vactst;
COMMIT;
+CREATE VIEW vactst_vacuum_counts AS
+ SELECT left(s.relname, 8), s.vacuum_count
+ FROM pg_stat_all_tables s
+ LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
+ WHERE c.relname = 'vactst' OR s.relname = 'vactst'
+ ORDER BY s.relname;
-- PROCESS_TOAST option
ALTER TABLE vactst ADD COLUMN t TEXT;
ALTER TABLE vactst ALTER COLUMN t SET STORAGE EXTERNAL;
+SELECT pg_stat_reset();
+ pg_stat_reset
+---------------
+
+(1 row)
+
VACUUM (PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+ left | vacuum_count
+----------+--------------
+ pg_toast | 0
+ vactst | 1
+(2 rows)
+
VACUUM (PROCESS_TOAST FALSE, FULL) vactst;
ERROR: PROCESS_TOAST required with VACUUM FULL
-- SKIP_DATABASE_STATS option
@@ -288,6 +307,47 @@ VACUUM (SKIP_DATABASE_STATS) vactst;
VACUUM (ONLY_DATABASE_STATS);
VACUUM (ONLY_DATABASE_STATS) vactst; -- error
ERROR: ONLY_DATABASE_STATS cannot be specified with a list of tables
+-- PROCESS_MAIN option
+SELECT pg_stat_reset();
+ pg_stat_reset
+---------------
+
+(1 row)
+
+VACUUM (PROCESS_MAIN FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+ left | vacuum_count
+----------+--------------
+ pg_toast | 1
+ vactst | 0
+(2 rows)
+
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+ left | vacuum_count
+----------+--------------
+ pg_toast | 1
+ vactst | 0
+(2 rows)
+
+SELECT relfilenode AS main_filenode FROM pg_class WHERE relname = 'vactst' \gset
+SELECT t.relfilenode AS toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst' \gset
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
+SELECT relfilenode = :main_filenode FROM pg_class WHERE relname = 'vactst';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT t.relfilenode = :toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst';
+ ?column?
+----------
+ f
+(1 row)
+
+DROP VIEW vactst_vacuum_counts;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 9da8f3e830..2985c1a99b 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -231,10 +231,19 @@ BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
ANALYZE vactst;
COMMIT;
+CREATE VIEW vactst_vacuum_counts AS
+ SELECT left(s.relname, 8), s.vacuum_count
+ FROM pg_stat_all_tables s
+ LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
+ WHERE c.relname = 'vactst' OR s.relname = 'vactst'
+ ORDER BY s.relname;
+
-- PROCESS_TOAST option
ALTER TABLE vactst ADD COLUMN t TEXT;
ALTER TABLE vactst ALTER COLUMN t SET STORAGE EXTERNAL;
+SELECT pg_stat_reset();
VACUUM (PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
VACUUM (PROCESS_TOAST FALSE, FULL) vactst;
-- SKIP_DATABASE_STATS option
@@ -244,6 +253,21 @@ VACUUM (SKIP_DATABASE_STATS) vactst;
VACUUM (ONLY_DATABASE_STATS);
VACUUM (ONLY_DATABASE_STATS) vactst; -- error
+-- PROCESS_MAIN option
+SELECT pg_stat_reset();
+VACUUM (PROCESS_MAIN FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+SELECT relfilenode AS main_filenode FROM pg_class WHERE relname = 'vactst' \gset
+SELECT t.relfilenode AS toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst' \gset
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
+SELECT relfilenode = :main_filenode FROM pg_class WHERE relname = 'vactst';
+SELECT t.relfilenode = :toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst';
+
+DROP VIEW vactst_vacuum_counts;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
--
2.25.1
On Wed, Mar 01, 2023 at 07:09:53PM +0100, Alvaro Herrera wrote:
Maybe instead of reading the log, read values from pg_stat_all_tables.
Ah, right. I was looking at pg_stat_user_tables yesterday, and forgot
that pg_stat_all_tables tracks toast tables, so it should be fine to
do some validation with that.
--
Michael
On Thu, Mar 2, 2023 at 4:13 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
On Wed, Mar 01, 2023 at 07:09:53PM +0100, Alvaro Herrera wrote:
On 2023-Mar-01, Michael Paquier wrote:
+-- PROCESS_MAIN option +VACUUM (PROCESS_MAIN FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst; +VACUUM (PROCESS_MAIN FALSE, FULL) vactst;Thinking a bit here. This set of tests does not make sure that the
main relation and/or the toast relation have been actually processed.
pg_stat_user_tables does not track what's happening on the toast
relations. So... What about adding some tests in 100_vacuumdb.pl
that rely on vacuumdb --verbose and check the logs produced? We
should make sure that the toast or the main relation are processed,
by tracking, for example, logs like vacuuming "schema.table". When
FULL is involved, we may want to track the changes on relfilenodes
depending on what's wanted.Maybe instead of reading the log, read values from pg_stat_all_tables.
Here is an attempt at that. Thanks for the idea.
I've reviewed the v4 patch. Here is a minor comment:
+SELECT * FROM vactst_vacuum_counts;
+ left | vacuum_count
+----------+--------------
+ pg_toast | 1
+ vactst | 0
+(2 rows)
+CREATE VIEW vactst_vacuum_counts AS
+ SELECT left(s.relname, 8), s.vacuum_count
+ FROM pg_stat_all_tables s
+ LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
+ WHERE c.relname = 'vactst' OR s.relname = 'vactst'
+ ORDER BY s.relname;
Cutting the toast relation name to 'pg_toast' is a bit confusing to me
as we have the pg_toast schema. How about using the following query
instead to improve the readability?
SELECT
CASE WHEN c.relname IS NULL THEN
s.relname
ELSE
'toast for ' || c.relname
END as relname,
s.vacuum_count
FROM pg_stat_all_tables s
LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
WHERE c.relname = 'vactst' OR s.relname = 'vactst'
We will get like:
SELECT * FROM vactst_vacuum_counts;
relname | vacuum_count
------------------+--------------
toast for vactst | 0
vactst | 1
(2 rows)
The rest looks good to me.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Mar 02, 2023 at 12:58:32PM +0900, Masahiko Sawada wrote:
Cutting the toast relation name to 'pg_toast' is a bit confusing to me
as we have the pg_toast schema. How about using the following query
instead to improve the readability?SELECT
CASE WHEN c.relname IS NULL THEN
s.relname
ELSE
'toast for ' || c.relname
END as relname,
s.vacuum_count
FROM pg_stat_all_tables s
LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
WHERE c.relname = 'vactst' OR s.relname = 'vactst'
Another tweak that I have learnt to like is to apply a filter with
regexp_replace(), see 090_reindexdb.pl:
regexp_replace(b.indname::text, '(pg_toast.pg_toast_)\\d+(_index)', '\\1<oid>\\2')
If you make that part of the view definition, the result is the same,
so that's up to which solution one prefers.
--
Michael
On Thu, Mar 02, 2023 at 02:21:08PM +0900, Michael Paquier wrote:
On Thu, Mar 02, 2023 at 12:58:32PM +0900, Masahiko Sawada wrote:
Cutting the toast relation name to 'pg_toast' is a bit confusing to me
as we have the pg_toast schema. How about using the following query
instead to improve the readability?SELECT
CASE WHEN c.relname IS NULL THEN
s.relname
ELSE
'toast for ' || c.relname
END as relname,
s.vacuum_count
FROM pg_stat_all_tables s
LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
WHERE c.relname = 'vactst' OR s.relname = 'vactst'Another tweak that I have learnt to like is to apply a filter with
regexp_replace(), see 090_reindexdb.pl:
regexp_replace(b.indname::text, '(pg_toast.pg_toast_)\\d+(_index)', '\\1<oid>\\2')If you make that part of the view definition, the result is the same,
so that's up to which solution one prefers.
Here's a new version of the patch that uses Sawada-san's suggestion.
Thanks for taking a look.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Attachments:
v5-0001-add-PROCESS_MAIN-to-VACUUM.patchtext/x-diff; charset=us-asciiDownload
From 221d46e7aacaf289ef001a6f61cc16c8e1046e2a Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathandbossart@gmail.com>
Date: Thu, 29 Dec 2022 15:31:49 -0800
Subject: [PATCH v5 1/1] add PROCESS_MAIN to VACUUM
---
doc/src/sgml/ref/vacuum.sgml | 13 ++++++
doc/src/sgml/ref/vacuumdb.sgml | 15 +++++++
src/backend/commands/vacuum.c | 28 ++++++++++---
src/backend/postmaster/autovacuum.c | 4 +-
src/bin/psql/tab-complete.c | 4 +-
src/bin/scripts/t/100_vacuumdb.pl | 7 ++++
src/bin/scripts/vacuumdb.c | 24 +++++++++++
src/include/commands/vacuum.h | 9 ++--
src/test/regress/expected/vacuum.out | 62 ++++++++++++++++++++++++++++
src/test/regress/sql/vacuum.sql | 26 ++++++++++++
10 files changed, 179 insertions(+), 13 deletions(-)
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index 545b23b54f..b6d30b5764 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -33,6 +33,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
DISABLE_PAGE_SKIPPING [ <replaceable class="parameter">boolean</replaceable> ]
SKIP_LOCKED [ <replaceable class="parameter">boolean</replaceable> ]
INDEX_CLEANUP { AUTO | ON | OFF }
+ PROCESS_MAIN [ <replaceable class="parameter">boolean</replaceable> ]
PROCESS_TOAST [ <replaceable class="parameter">boolean</replaceable> ]
TRUNCATE [ <replaceable class="parameter">boolean</replaceable> ]
PARALLEL <replaceable class="parameter">integer</replaceable>
@@ -238,6 +239,18 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>PROCESS_MAIN</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should attempt to process the
+ main relation. This is usually the desired behavior and is the default.
+ Setting this option to false may be useful when it is only necessary to
+ vacuum a relation's corresponding <literal>TOAST</literal> table.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>PROCESS_TOAST</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/vacuumdb.sgml b/doc/src/sgml/ref/vacuumdb.sgml
index 841aced3bd..74bac2d4ba 100644
--- a/doc/src/sgml/ref/vacuumdb.sgml
+++ b/doc/src/sgml/ref/vacuumdb.sgml
@@ -317,6 +317,21 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--no-process-main</option></term>
+ <listitem>
+ <para>
+ Skip the main relation.
+ </para>
+ <note>
+ <para>
+ This option is only available for servers running
+ <productname>PostgreSQL</productname> 16 and later.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--no-process-toast</option></term>
<listitem>
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index aa79d9de4d..88ac1df2b5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -115,6 +115,7 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
bool freeze = false;
bool full = false;
bool disable_page_skipping = false;
+ bool process_main = true;
bool process_toast = true;
bool skip_database_stats = false;
bool only_database_stats = false;
@@ -168,6 +169,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
params.index_cleanup = get_vacoptval_from_boolean(opt);
}
}
+ else if (strcmp(opt->defname, "process_main") == 0)
+ process_main = defGetBoolean(opt);
else if (strcmp(opt->defname, "process_toast") == 0)
process_toast = defGetBoolean(opt);
else if (strcmp(opt->defname, "truncate") == 0)
@@ -226,7 +229,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
(disable_page_skipping ? VACOPT_DISABLE_PAGE_SKIPPING : 0) |
(process_toast ? VACOPT_PROCESS_TOAST : 0) |
(skip_database_stats ? VACOPT_SKIP_DATABASE_STATS : 0) |
- (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0);
+ (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0) |
+ (process_main ? VACOPT_PROCESS_MAIN : 0);
/* sanity checks on options */
Assert(params.options & (VACOPT_VACUUM | VACOPT_ANALYZE));
@@ -367,9 +371,10 @@ vacuum(List *relations, VacuumParams *params,
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ONLY_DATABASE_STATS cannot be specified with a list of tables")));
- /* don't require people to turn off PROCESS_TOAST explicitly */
+ /* don't require people to turn off PROCESS_TOAST/MAIN explicitly */
if (params->options & ~(VACOPT_VACUUM |
VACOPT_VERBOSE |
+ VACOPT_PROCESS_MAIN |
VACOPT_PROCESS_TOAST |
VACOPT_ONLY_DATABASE_STATS))
ereport(ERROR,
@@ -2031,10 +2036,12 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/*
* Remember the relation's TOAST relation for later, if the caller asked
* us to process it. In VACUUM FULL, though, the toast table is
- * automatically rebuilt by cluster_rel so we shouldn't recurse to it.
+ * automatically rebuilt by cluster_rel so we shouldn't recurse to it
+ * unless PROCESS_MAIN is disabled.
*/
if ((params->options & VACOPT_PROCESS_TOAST) != 0 &&
- (params->options & VACOPT_FULL) == 0)
+ ((params->options & VACOPT_FULL) == 0 ||
+ (params->options & VACOPT_PROCESS_MAIN) == 0))
toast_relid = rel->rd_rel->reltoastrelid;
else
toast_relid = InvalidOid;
@@ -2053,7 +2060,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/*
* Do the actual work --- either FULL or "lazy" vacuum
*/
- if (params->options & VACOPT_FULL)
+ if (params->options & VACOPT_FULL &&
+ params->options & VACOPT_PROCESS_MAIN)
{
ClusterParams cluster_params = {0};
@@ -2067,7 +2075,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
cluster_rel(relid, InvalidOid, &cluster_params);
}
- else
+ else if (params->options & VACOPT_PROCESS_MAIN)
table_relation_vacuum(rel, params, vac_strategy);
/* Roll back any GUC changes executed by index functions */
@@ -2094,7 +2102,15 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
* totally unimportant for toast relations.
*/
if (toast_relid != InvalidOid)
+ {
+ /* we force VACOPT_PROCESS_MAIN so vacuum_rel() processes it */
+ bool force_opt = ((params->options & VACOPT_PROCESS_MAIN) == 0);
+
+ params->options |= VACOPT_PROCESS_MAIN;
vacuum_rel(toast_relid, NULL, params, true);
+ if (force_opt)
+ params->options &= ~VACOPT_PROCESS_MAIN;
+ }
/*
* Now release the session-level lock on the main table.
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index ff6149a179..c0e2e00a7e 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -2860,7 +2860,9 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
* skip vac_update_datfrozenxid(); we'll do that separately.
*/
tab->at_params.options =
- (dovacuum ? (VACOPT_VACUUM | VACOPT_SKIP_DATABASE_STATS) : 0) |
+ (dovacuum ? (VACOPT_VACUUM |
+ VACOPT_PROCESS_MAIN |
+ VACOPT_SKIP_DATABASE_STATS) : 0) |
(doanalyze ? VACOPT_ANALYZE : 0) |
(!wraparound ? VACOPT_SKIP_LOCKED : 0);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5e1882eaea..8f12af799b 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -4618,10 +4618,10 @@ psql_completion(const char *text, int start, int end)
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
COMPLETE_WITH("FULL", "FREEZE", "ANALYZE", "VERBOSE",
"DISABLE_PAGE_SKIPPING", "SKIP_LOCKED",
- "INDEX_CLEANUP", "PROCESS_TOAST",
+ "INDEX_CLEANUP", "PROCESS_MAIN", "PROCESS_TOAST",
"TRUNCATE", "PARALLEL", "SKIP_DATABASE_STATS",
"ONLY_DATABASE_STATS");
- else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS"))
+ else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_MAIN|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS"))
COMPLETE_WITH("ON", "OFF");
else if (TailMatches("INDEX_CLEANUP"))
COMPLETE_WITH("AUTO", "ON", "OFF");
diff --git a/src/bin/scripts/t/100_vacuumdb.pl b/src/bin/scripts/t/100_vacuumdb.pl
index 3cfbaaec0d..46101899ae 100644
--- a/src/bin/scripts/t/100_vacuumdb.pl
+++ b/src/bin/scripts/t/100_vacuumdb.pl
@@ -65,6 +65,13 @@ $node->issues_sql_like(
$node->command_fails(
[ 'vacuumdb', '--analyze-only', '--no-truncate', 'postgres' ],
'--analyze-only and --no-truncate specified together');
+$node->issues_sql_like(
+ [ 'vacuumdb', '--no-process-main', 'postgres' ],
+ qr/statement: VACUUM \(PROCESS_MAIN FALSE, SKIP_DATABASE_STATS\).*;/,
+ 'vacuumdb --no-process-main');
+$node->command_fails(
+ [ 'vacuumdb', '--analyze-only', '--no-process-main', 'postgres' ],
+ '--analyze-only and --no-process_main specified together');
$node->issues_sql_like(
[ 'vacuumdb', '--no-process-toast', 'postgres' ],
qr/statement: VACUUM \(PROCESS_TOAST FALSE, SKIP_DATABASE_STATS\).*;/,
diff --git a/src/bin/scripts/vacuumdb.c b/src/bin/scripts/vacuumdb.c
index 58b894216b..39be265b5b 100644
--- a/src/bin/scripts/vacuumdb.c
+++ b/src/bin/scripts/vacuumdb.c
@@ -43,6 +43,7 @@ typedef struct vacuumingOptions
bool no_index_cleanup;
bool force_index_cleanup;
bool do_truncate;
+ bool process_main;
bool process_toast;
bool skip_database_stats;
} vacuumingOptions;
@@ -121,6 +122,7 @@ main(int argc, char *argv[])
{"force-index-cleanup", no_argument, NULL, 9},
{"no-truncate", no_argument, NULL, 10},
{"no-process-toast", no_argument, NULL, 11},
+ {"no-process-main", no_argument, NULL, 12},
{NULL, 0, NULL, 0}
};
@@ -148,6 +150,7 @@ main(int argc, char *argv[])
vacopts.no_index_cleanup = false;
vacopts.force_index_cleanup = false;
vacopts.do_truncate = true;
+ vacopts.process_main = true;
vacopts.process_toast = true;
pg_logging_init(argv[0]);
@@ -260,6 +263,9 @@ main(int argc, char *argv[])
case 11:
vacopts.process_toast = false;
break;
+ case 12:
+ vacopts.process_main = false;
+ break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -312,6 +318,9 @@ main(int argc, char *argv[])
if (!vacopts.do_truncate)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-truncate");
+ if (!vacopts.process_main)
+ pg_fatal("cannot use the \"%s\" option when performing only analyze",
+ "no-process-main");
if (!vacopts.process_toast)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-process-toast");
@@ -508,6 +517,13 @@ vacuum_one_database(ConnParams *cparams,
"no-truncate", "12");
}
+ if (!vacopts->process_main && PQserverVersion(conn) < 160000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "no-process-main", "16");
+ }
+
if (!vacopts->process_toast && PQserverVersion(conn) < 140000)
{
PQfinish(conn);
@@ -976,6 +992,13 @@ prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
appendPQExpBuffer(sql, "%sTRUNCATE FALSE", sep);
sep = comma;
}
+ if (!vacopts->process_main)
+ {
+ /* PROCESS_MAIN is supported since v16 */
+ Assert(serverVersion >= 160000);
+ appendPQExpBuffer(sql, "%sPROCESS_MAIN FALSE", sep);
+ sep = comma;
+ }
if (!vacopts->process_toast)
{
/* PROCESS_TOAST is supported since v14 */
@@ -1090,6 +1113,7 @@ help(const char *progname)
printf(_(" --min-mxid-age=MXID_AGE minimum multixact ID age of tables to vacuum\n"));
printf(_(" --min-xid-age=XID_AGE minimum transaction ID age of tables to vacuum\n"));
printf(_(" --no-index-cleanup don't remove index entries that point to dead tuples\n"));
+ printf(_(" --no-process-main skip the main relation\n"));
printf(_(" --no-process-toast skip the TOAST table associated with the table to vacuum\n"));
printf(_(" --no-truncate don't truncate empty pages at the end of the table\n"));
printf(_(" -n, --schema=PATTERN vacuum tables in the specified schema(s) only\n"));
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 689dbb7702..bdfd96cfec 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -186,10 +186,11 @@ typedef struct VacAttrStats
#define VACOPT_FREEZE 0x08 /* FREEZE option */
#define VACOPT_FULL 0x10 /* FULL (non-concurrent) vacuum */
#define VACOPT_SKIP_LOCKED 0x20 /* skip if cannot get lock */
-#define VACOPT_PROCESS_TOAST 0x40 /* process the TOAST table, if any */
-#define VACOPT_DISABLE_PAGE_SKIPPING 0x80 /* don't skip any pages */
-#define VACOPT_SKIP_DATABASE_STATS 0x100 /* skip vac_update_datfrozenxid() */
-#define VACOPT_ONLY_DATABASE_STATS 0x200 /* only vac_update_datfrozenxid() */
+#define VACOPT_PROCESS_MAIN 0x40 /* process main relation */
+#define VACOPT_PROCESS_TOAST 0x80 /* process the TOAST table, if any */
+#define VACOPT_DISABLE_PAGE_SKIPPING 0x100 /* don't skip any pages */
+#define VACOPT_SKIP_DATABASE_STATS 0x200 /* skip vac_update_datfrozenxid() */
+#define VACOPT_ONLY_DATABASE_STATS 0x400 /* only vac_update_datfrozenxid() */
/*
* Values used by index_cleanup and truncate params.
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index 458adee7f8..2d82b13fa7 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -276,10 +276,31 @@ RESET default_transaction_isolation;
BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
ANALYZE vactst;
COMMIT;
+CREATE VIEW vactst_vacuum_counts AS
+ SELECT CASE WHEN c.relname IS NULL THEN s.relname
+ ELSE 'toast for ' || c.relname END as rel,
+ s.vacuum_count
+ FROM pg_stat_all_tables s
+ LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
+ WHERE c.relname = 'vactst' OR s.relname = 'vactst'
+ ORDER BY rel;
-- PROCESS_TOAST option
ALTER TABLE vactst ADD COLUMN t TEXT;
ALTER TABLE vactst ALTER COLUMN t SET STORAGE EXTERNAL;
+SELECT pg_stat_reset();
+ pg_stat_reset
+---------------
+
+(1 row)
+
VACUUM (PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+ rel | vacuum_count
+------------------+--------------
+ toast for vactst | 0
+ vactst | 1
+(2 rows)
+
VACUUM (PROCESS_TOAST FALSE, FULL) vactst;
ERROR: PROCESS_TOAST required with VACUUM FULL
-- SKIP_DATABASE_STATS option
@@ -288,6 +309,47 @@ VACUUM (SKIP_DATABASE_STATS) vactst;
VACUUM (ONLY_DATABASE_STATS);
VACUUM (ONLY_DATABASE_STATS) vactst; -- error
ERROR: ONLY_DATABASE_STATS cannot be specified with a list of tables
+-- PROCESS_MAIN option
+SELECT pg_stat_reset();
+ pg_stat_reset
+---------------
+
+(1 row)
+
+VACUUM (PROCESS_MAIN FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+ rel | vacuum_count
+------------------+--------------
+ toast for vactst | 1
+ vactst | 0
+(2 rows)
+
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+ rel | vacuum_count
+------------------+--------------
+ toast for vactst | 1
+ vactst | 0
+(2 rows)
+
+SELECT relfilenode AS main_filenode FROM pg_class WHERE relname = 'vactst' \gset
+SELECT t.relfilenode AS toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst' \gset
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
+SELECT relfilenode = :main_filenode FROM pg_class WHERE relname = 'vactst';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT t.relfilenode = :toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst';
+ ?column?
+----------
+ f
+(1 row)
+
+DROP VIEW vactst_vacuum_counts;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 9da8f3e830..a856aa2ccf 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -231,10 +231,21 @@ BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
ANALYZE vactst;
COMMIT;
+CREATE VIEW vactst_vacuum_counts AS
+ SELECT CASE WHEN c.relname IS NULL THEN s.relname
+ ELSE 'toast for ' || c.relname END as rel,
+ s.vacuum_count
+ FROM pg_stat_all_tables s
+ LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
+ WHERE c.relname = 'vactst' OR s.relname = 'vactst'
+ ORDER BY rel;
+
-- PROCESS_TOAST option
ALTER TABLE vactst ADD COLUMN t TEXT;
ALTER TABLE vactst ALTER COLUMN t SET STORAGE EXTERNAL;
+SELECT pg_stat_reset();
VACUUM (PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
VACUUM (PROCESS_TOAST FALSE, FULL) vactst;
-- SKIP_DATABASE_STATS option
@@ -244,6 +255,21 @@ VACUUM (SKIP_DATABASE_STATS) vactst;
VACUUM (ONLY_DATABASE_STATS);
VACUUM (ONLY_DATABASE_STATS) vactst; -- error
+-- PROCESS_MAIN option
+SELECT pg_stat_reset();
+VACUUM (PROCESS_MAIN FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+SELECT relfilenode AS main_filenode FROM pg_class WHERE relname = 'vactst' \gset
+SELECT t.relfilenode AS toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst' \gset
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
+SELECT relfilenode = :main_filenode FROM pg_class WHERE relname = 'vactst';
+SELECT t.relfilenode = :toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst';
+
+DROP VIEW vactst_vacuum_counts;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
--
2.25.1
On Thu, Mar 2, 2023 at 2:26 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
On Thu, Mar 02, 2023 at 02:21:08PM +0900, Michael Paquier wrote:
On Thu, Mar 02, 2023 at 12:58:32PM +0900, Masahiko Sawada wrote:
Cutting the toast relation name to 'pg_toast' is a bit confusing to me
as we have the pg_toast schema. How about using the following query
instead to improve the readability?SELECT
CASE WHEN c.relname IS NULL THEN
s.relname
ELSE
'toast for ' || c.relname
END as relname,
s.vacuum_count
FROM pg_stat_all_tables s
LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
WHERE c.relname = 'vactst' OR s.relname = 'vactst'Another tweak that I have learnt to like is to apply a filter with
regexp_replace(), see 090_reindexdb.pl:
regexp_replace(b.indname::text, '(pg_toast.pg_toast_)\\d+(_index)', '\\1<oid>\\2')If you make that part of the view definition, the result is the same,
so that's up to which solution one prefers.Here's a new version of the patch that uses Sawada-san's suggestion.
Thanks for taking a look.
Thank you for updating the patch. The patch looks good to me.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Wed, Mar 01, 2023 at 09:26:37AM -0800, Nathan Bossart wrote:
Thanks for taking a look.
On Wed, Mar 01, 2023 at 03:31:48PM +0900, Michael Paquier wrote:
PROCESS_TOAST has that: /* sanity check for PROCESS_TOAST */ if ((params->options & VACOPT_FULL) != 0 && (params->options & VACOPT_PROCESS_TOAST) == 0) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("PROCESS_TOAST required with VACUUM FULL"))); [...] - if (params->options & VACOPT_FULL) + if (params->options & VACOPT_FULL && + params->options & VACOPT_PROCESS_MAIN) {Shouldn't we apply the same rule for PROCESS_MAIN? One of the
regression tests added means that FULL takes priority over
PROCESS_MAIN=FALSE, which is a bit confusing IMO.I don't think so. We disallow FULL without PROCESS_TOAST because there
presently isn't a way to VACUUM FULL the main relation without rebuilding
its TOAST table. However, FULL without PROCESS_MAIN can be used to run
VACUUM FULL on only the TOAST table.
- if (params->options & VACOPT_FULL)
+ if (params->options & VACOPT_FULL &&
+ params->options & VACOPT_PROCESS_MAIN)
Perhaps this is a bit under-parenthesized, while reading through it
once again..
+ {
+ /* we force VACOPT_PROCESS_MAIN so vacuum_rel() processes it */
+ bool force_opt = ((params->options & VACOPT_PROCESS_MAIN) == 0);
+
+ params->options |= VACOPT_PROCESS_MAIN;
vacuum_rel(toast_relid, NULL, params, true);
+ if (force_opt)
+ params->options &= ~VACOPT_PROCESS_MAIN;
Zigzagging with the centralized VacuumParams is a bit inelegant.
Perhaps it would be neater to copy VacuumParams and append
VACOPT_PROCESS_MAIN on the copy?
An extra question: should we check the behavior of the commands when
applying a list of relations to VACUUM?
--
Michael
On Thu, Mar 02, 2023 at 02:53:02PM +0900, Michael Paquier wrote:
- if (params->options & VACOPT_FULL) + if (params->options & VACOPT_FULL && + params->options & VACOPT_PROCESS_MAIN) Perhaps this is a bit under-parenthesized, while reading through it once again..
fixed
+ { + /* we force VACOPT_PROCESS_MAIN so vacuum_rel() processes it */ + bool force_opt = ((params->options & VACOPT_PROCESS_MAIN) == 0); + + params->options |= VACOPT_PROCESS_MAIN; vacuum_rel(toast_relid, NULL, params, true); + if (force_opt) + params->options &= ~VACOPT_PROCESS_MAIN; Zigzagging with the centralized VacuumParams is a bit inelegant. Perhaps it would be neater to copy VacuumParams and append VACOPT_PROCESS_MAIN on the copy?
done
An extra question: should we check the behavior of the commands when
applying a list of relations to VACUUM?
I don't feel a strong need for that, especially now that we aren't
modifying params anymore.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Attachments:
v6-0001-add-PROCESS_MAIN-to-VACUUM.patchtext/x-diff; charset=us-asciiDownload
From 0a4563a536fc4aa044d6d1b663ce28b6ef0c36f7 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathandbossart@gmail.com>
Date: Thu, 29 Dec 2022 15:31:49 -0800
Subject: [PATCH v6 1/1] add PROCESS_MAIN to VACUUM
---
doc/src/sgml/ref/vacuum.sgml | 13 ++++++
doc/src/sgml/ref/vacuumdb.sgml | 15 +++++++
src/backend/commands/vacuum.c | 30 ++++++++++----
src/backend/postmaster/autovacuum.c | 4 +-
src/bin/psql/tab-complete.c | 4 +-
src/bin/scripts/t/100_vacuumdb.pl | 7 ++++
src/bin/scripts/vacuumdb.c | 24 +++++++++++
src/include/commands/vacuum.h | 9 ++--
src/test/regress/expected/vacuum.out | 61 ++++++++++++++++++++++++++++
src/test/regress/sql/vacuum.sql | 25 ++++++++++++
10 files changed, 178 insertions(+), 14 deletions(-)
diff --git a/doc/src/sgml/ref/vacuum.sgml b/doc/src/sgml/ref/vacuum.sgml
index 545b23b54f..b6d30b5764 100644
--- a/doc/src/sgml/ref/vacuum.sgml
+++ b/doc/src/sgml/ref/vacuum.sgml
@@ -33,6 +33,7 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
DISABLE_PAGE_SKIPPING [ <replaceable class="parameter">boolean</replaceable> ]
SKIP_LOCKED [ <replaceable class="parameter">boolean</replaceable> ]
INDEX_CLEANUP { AUTO | ON | OFF }
+ PROCESS_MAIN [ <replaceable class="parameter">boolean</replaceable> ]
PROCESS_TOAST [ <replaceable class="parameter">boolean</replaceable> ]
TRUNCATE [ <replaceable class="parameter">boolean</replaceable> ]
PARALLEL <replaceable class="parameter">integer</replaceable>
@@ -238,6 +239,18 @@ VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <replaceable class="paramet
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>PROCESS_MAIN</literal></term>
+ <listitem>
+ <para>
+ Specifies that <command>VACUUM</command> should attempt to process the
+ main relation. This is usually the desired behavior and is the default.
+ Setting this option to false may be useful when it is only necessary to
+ vacuum a relation's corresponding <literal>TOAST</literal> table.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>PROCESS_TOAST</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/vacuumdb.sgml b/doc/src/sgml/ref/vacuumdb.sgml
index 841aced3bd..74bac2d4ba 100644
--- a/doc/src/sgml/ref/vacuumdb.sgml
+++ b/doc/src/sgml/ref/vacuumdb.sgml
@@ -317,6 +317,21 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--no-process-main</option></term>
+ <listitem>
+ <para>
+ Skip the main relation.
+ </para>
+ <note>
+ <para>
+ This option is only available for servers running
+ <productname>PostgreSQL</productname> 16 and later.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--no-process-toast</option></term>
<listitem>
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index aa79d9de4d..7633fa249c 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -115,6 +115,7 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
bool freeze = false;
bool full = false;
bool disable_page_skipping = false;
+ bool process_main = true;
bool process_toast = true;
bool skip_database_stats = false;
bool only_database_stats = false;
@@ -168,6 +169,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
params.index_cleanup = get_vacoptval_from_boolean(opt);
}
}
+ else if (strcmp(opt->defname, "process_main") == 0)
+ process_main = defGetBoolean(opt);
else if (strcmp(opt->defname, "process_toast") == 0)
process_toast = defGetBoolean(opt);
else if (strcmp(opt->defname, "truncate") == 0)
@@ -226,7 +229,8 @@ ExecVacuum(ParseState *pstate, VacuumStmt *vacstmt, bool isTopLevel)
(disable_page_skipping ? VACOPT_DISABLE_PAGE_SKIPPING : 0) |
(process_toast ? VACOPT_PROCESS_TOAST : 0) |
(skip_database_stats ? VACOPT_SKIP_DATABASE_STATS : 0) |
- (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0);
+ (only_database_stats ? VACOPT_ONLY_DATABASE_STATS : 0) |
+ (process_main ? VACOPT_PROCESS_MAIN : 0);
/* sanity checks on options */
Assert(params.options & (VACOPT_VACUUM | VACOPT_ANALYZE));
@@ -367,9 +371,10 @@ vacuum(List *relations, VacuumParams *params,
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ONLY_DATABASE_STATS cannot be specified with a list of tables")));
- /* don't require people to turn off PROCESS_TOAST explicitly */
+ /* don't require people to turn off PROCESS_TOAST/MAIN explicitly */
if (params->options & ~(VACOPT_VACUUM |
VACOPT_VERBOSE |
+ VACOPT_PROCESS_MAIN |
VACOPT_PROCESS_TOAST |
VACOPT_ONLY_DATABASE_STATS))
ereport(ERROR,
@@ -2031,10 +2036,12 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/*
* Remember the relation's TOAST relation for later, if the caller asked
* us to process it. In VACUUM FULL, though, the toast table is
- * automatically rebuilt by cluster_rel so we shouldn't recurse to it.
+ * automatically rebuilt by cluster_rel so we shouldn't recurse to it
+ * unless PROCESS_MAIN is disabled.
*/
if ((params->options & VACOPT_PROCESS_TOAST) != 0 &&
- (params->options & VACOPT_FULL) == 0)
+ ((params->options & VACOPT_FULL) == 0 ||
+ (params->options & VACOPT_PROCESS_MAIN) == 0))
toast_relid = rel->rd_rel->reltoastrelid;
else
toast_relid = InvalidOid;
@@ -2053,7 +2060,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/*
* Do the actual work --- either FULL or "lazy" vacuum
*/
- if (params->options & VACOPT_FULL)
+ if ((params->options & VACOPT_FULL) &&
+ (params->options & VACOPT_PROCESS_MAIN))
{
ClusterParams cluster_params = {0};
@@ -2067,7 +2075,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
cluster_rel(relid, InvalidOid, &cluster_params);
}
- else
+ else if (params->options & VACOPT_PROCESS_MAIN)
table_relation_vacuum(rel, params, vac_strategy);
/* Roll back any GUC changes executed by index functions */
@@ -2094,7 +2102,15 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
* totally unimportant for toast relations.
*/
if (toast_relid != InvalidOid)
- vacuum_rel(toast_relid, NULL, params, true);
+ {
+ VacuumParams toast_params;
+
+ /* force VACOPT_PROCESS_MAIN so vacuum_rel() processes it */
+ memcpy(&toast_params, params, sizeof(VacuumParams));
+ toast_params.options |= VACOPT_PROCESS_MAIN;
+
+ vacuum_rel(toast_relid, NULL, &toast_params, true);
+ }
/*
* Now release the session-level lock on the main table.
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index ff6149a179..c0e2e00a7e 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -2860,7 +2860,9 @@ table_recheck_autovac(Oid relid, HTAB *table_toast_map,
* skip vac_update_datfrozenxid(); we'll do that separately.
*/
tab->at_params.options =
- (dovacuum ? (VACOPT_VACUUM | VACOPT_SKIP_DATABASE_STATS) : 0) |
+ (dovacuum ? (VACOPT_VACUUM |
+ VACOPT_PROCESS_MAIN |
+ VACOPT_SKIP_DATABASE_STATS) : 0) |
(doanalyze ? VACOPT_ANALYZE : 0) |
(!wraparound ? VACOPT_SKIP_LOCKED : 0);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5e1882eaea..8f12af799b 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -4618,10 +4618,10 @@ psql_completion(const char *text, int start, int end)
if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
COMPLETE_WITH("FULL", "FREEZE", "ANALYZE", "VERBOSE",
"DISABLE_PAGE_SKIPPING", "SKIP_LOCKED",
- "INDEX_CLEANUP", "PROCESS_TOAST",
+ "INDEX_CLEANUP", "PROCESS_MAIN", "PROCESS_TOAST",
"TRUNCATE", "PARALLEL", "SKIP_DATABASE_STATS",
"ONLY_DATABASE_STATS");
- else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS"))
+ else if (TailMatches("FULL|FREEZE|ANALYZE|VERBOSE|DISABLE_PAGE_SKIPPING|SKIP_LOCKED|PROCESS_MAIN|PROCESS_TOAST|TRUNCATE|SKIP_DATABASE_STATS|ONLY_DATABASE_STATS"))
COMPLETE_WITH("ON", "OFF");
else if (TailMatches("INDEX_CLEANUP"))
COMPLETE_WITH("AUTO", "ON", "OFF");
diff --git a/src/bin/scripts/t/100_vacuumdb.pl b/src/bin/scripts/t/100_vacuumdb.pl
index 3cfbaaec0d..46101899ae 100644
--- a/src/bin/scripts/t/100_vacuumdb.pl
+++ b/src/bin/scripts/t/100_vacuumdb.pl
@@ -65,6 +65,13 @@ $node->issues_sql_like(
$node->command_fails(
[ 'vacuumdb', '--analyze-only', '--no-truncate', 'postgres' ],
'--analyze-only and --no-truncate specified together');
+$node->issues_sql_like(
+ [ 'vacuumdb', '--no-process-main', 'postgres' ],
+ qr/statement: VACUUM \(PROCESS_MAIN FALSE, SKIP_DATABASE_STATS\).*;/,
+ 'vacuumdb --no-process-main');
+$node->command_fails(
+ [ 'vacuumdb', '--analyze-only', '--no-process-main', 'postgres' ],
+ '--analyze-only and --no-process_main specified together');
$node->issues_sql_like(
[ 'vacuumdb', '--no-process-toast', 'postgres' ],
qr/statement: VACUUM \(PROCESS_TOAST FALSE, SKIP_DATABASE_STATS\).*;/,
diff --git a/src/bin/scripts/vacuumdb.c b/src/bin/scripts/vacuumdb.c
index 58b894216b..39be265b5b 100644
--- a/src/bin/scripts/vacuumdb.c
+++ b/src/bin/scripts/vacuumdb.c
@@ -43,6 +43,7 @@ typedef struct vacuumingOptions
bool no_index_cleanup;
bool force_index_cleanup;
bool do_truncate;
+ bool process_main;
bool process_toast;
bool skip_database_stats;
} vacuumingOptions;
@@ -121,6 +122,7 @@ main(int argc, char *argv[])
{"force-index-cleanup", no_argument, NULL, 9},
{"no-truncate", no_argument, NULL, 10},
{"no-process-toast", no_argument, NULL, 11},
+ {"no-process-main", no_argument, NULL, 12},
{NULL, 0, NULL, 0}
};
@@ -148,6 +150,7 @@ main(int argc, char *argv[])
vacopts.no_index_cleanup = false;
vacopts.force_index_cleanup = false;
vacopts.do_truncate = true;
+ vacopts.process_main = true;
vacopts.process_toast = true;
pg_logging_init(argv[0]);
@@ -260,6 +263,9 @@ main(int argc, char *argv[])
case 11:
vacopts.process_toast = false;
break;
+ case 12:
+ vacopts.process_main = false;
+ break;
default:
/* getopt_long already emitted a complaint */
pg_log_error_hint("Try \"%s --help\" for more information.", progname);
@@ -312,6 +318,9 @@ main(int argc, char *argv[])
if (!vacopts.do_truncate)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-truncate");
+ if (!vacopts.process_main)
+ pg_fatal("cannot use the \"%s\" option when performing only analyze",
+ "no-process-main");
if (!vacopts.process_toast)
pg_fatal("cannot use the \"%s\" option when performing only analyze",
"no-process-toast");
@@ -508,6 +517,13 @@ vacuum_one_database(ConnParams *cparams,
"no-truncate", "12");
}
+ if (!vacopts->process_main && PQserverVersion(conn) < 160000)
+ {
+ PQfinish(conn);
+ pg_fatal("cannot use the \"%s\" option on server versions older than PostgreSQL %s",
+ "no-process-main", "16");
+ }
+
if (!vacopts->process_toast && PQserverVersion(conn) < 140000)
{
PQfinish(conn);
@@ -976,6 +992,13 @@ prepare_vacuum_command(PQExpBuffer sql, int serverVersion,
appendPQExpBuffer(sql, "%sTRUNCATE FALSE", sep);
sep = comma;
}
+ if (!vacopts->process_main)
+ {
+ /* PROCESS_MAIN is supported since v16 */
+ Assert(serverVersion >= 160000);
+ appendPQExpBuffer(sql, "%sPROCESS_MAIN FALSE", sep);
+ sep = comma;
+ }
if (!vacopts->process_toast)
{
/* PROCESS_TOAST is supported since v14 */
@@ -1090,6 +1113,7 @@ help(const char *progname)
printf(_(" --min-mxid-age=MXID_AGE minimum multixact ID age of tables to vacuum\n"));
printf(_(" --min-xid-age=XID_AGE minimum transaction ID age of tables to vacuum\n"));
printf(_(" --no-index-cleanup don't remove index entries that point to dead tuples\n"));
+ printf(_(" --no-process-main skip the main relation\n"));
printf(_(" --no-process-toast skip the TOAST table associated with the table to vacuum\n"));
printf(_(" --no-truncate don't truncate empty pages at the end of the table\n"));
printf(_(" -n, --schema=PATTERN vacuum tables in the specified schema(s) only\n"));
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 689dbb7702..bdfd96cfec 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -186,10 +186,11 @@ typedef struct VacAttrStats
#define VACOPT_FREEZE 0x08 /* FREEZE option */
#define VACOPT_FULL 0x10 /* FULL (non-concurrent) vacuum */
#define VACOPT_SKIP_LOCKED 0x20 /* skip if cannot get lock */
-#define VACOPT_PROCESS_TOAST 0x40 /* process the TOAST table, if any */
-#define VACOPT_DISABLE_PAGE_SKIPPING 0x80 /* don't skip any pages */
-#define VACOPT_SKIP_DATABASE_STATS 0x100 /* skip vac_update_datfrozenxid() */
-#define VACOPT_ONLY_DATABASE_STATS 0x200 /* only vac_update_datfrozenxid() */
+#define VACOPT_PROCESS_MAIN 0x40 /* process main relation */
+#define VACOPT_PROCESS_TOAST 0x80 /* process the TOAST table, if any */
+#define VACOPT_DISABLE_PAGE_SKIPPING 0x100 /* don't skip any pages */
+#define VACOPT_SKIP_DATABASE_STATS 0x200 /* skip vac_update_datfrozenxid() */
+#define VACOPT_ONLY_DATABASE_STATS 0x400 /* only vac_update_datfrozenxid() */
/*
* Values used by index_cleanup and truncate params.
diff --git a/src/test/regress/expected/vacuum.out b/src/test/regress/expected/vacuum.out
index 458adee7f8..50c41984da 100644
--- a/src/test/regress/expected/vacuum.out
+++ b/src/test/regress/expected/vacuum.out
@@ -276,10 +276,30 @@ RESET default_transaction_isolation;
BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
ANALYZE vactst;
COMMIT;
+CREATE VIEW vactst_vacuum_counts AS
+ SELECT CASE WHEN c.relname IS NULL THEN 'main' ELSE 'toast' END as rel,
+ s.vacuum_count
+ FROM pg_stat_all_tables s
+ LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
+ WHERE c.relname = 'vactst' OR s.relname = 'vactst'
+ ORDER BY rel;
-- PROCESS_TOAST option
ALTER TABLE vactst ADD COLUMN t TEXT;
ALTER TABLE vactst ALTER COLUMN t SET STORAGE EXTERNAL;
+SELECT pg_stat_reset();
+ pg_stat_reset
+---------------
+
+(1 row)
+
VACUUM (PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+ rel | vacuum_count
+-------+--------------
+ main | 1
+ toast | 0
+(2 rows)
+
VACUUM (PROCESS_TOAST FALSE, FULL) vactst;
ERROR: PROCESS_TOAST required with VACUUM FULL
-- SKIP_DATABASE_STATS option
@@ -288,6 +308,47 @@ VACUUM (SKIP_DATABASE_STATS) vactst;
VACUUM (ONLY_DATABASE_STATS);
VACUUM (ONLY_DATABASE_STATS) vactst; -- error
ERROR: ONLY_DATABASE_STATS cannot be specified with a list of tables
+-- PROCESS_MAIN option
+SELECT pg_stat_reset();
+ pg_stat_reset
+---------------
+
+(1 row)
+
+VACUUM (PROCESS_MAIN FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+ rel | vacuum_count
+-------+--------------
+ main | 0
+ toast | 1
+(2 rows)
+
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+ rel | vacuum_count
+-------+--------------
+ main | 0
+ toast | 1
+(2 rows)
+
+SELECT relfilenode AS main_filenode FROM pg_class WHERE relname = 'vactst' \gset
+SELECT t.relfilenode AS toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst' \gset
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
+SELECT relfilenode = :main_filenode FROM pg_class WHERE relname = 'vactst';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT t.relfilenode = :toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst';
+ ?column?
+----------
+ f
+(1 row)
+
+DROP VIEW vactst_vacuum_counts;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
diff --git a/src/test/regress/sql/vacuum.sql b/src/test/regress/sql/vacuum.sql
index 9da8f3e830..f1329c0db9 100644
--- a/src/test/regress/sql/vacuum.sql
+++ b/src/test/regress/sql/vacuum.sql
@@ -231,10 +231,20 @@ BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
ANALYZE vactst;
COMMIT;
+CREATE VIEW vactst_vacuum_counts AS
+ SELECT CASE WHEN c.relname IS NULL THEN 'main' ELSE 'toast' END as rel,
+ s.vacuum_count
+ FROM pg_stat_all_tables s
+ LEFT JOIN pg_class c ON s.relid = c.reltoastrelid
+ WHERE c.relname = 'vactst' OR s.relname = 'vactst'
+ ORDER BY rel;
+
-- PROCESS_TOAST option
ALTER TABLE vactst ADD COLUMN t TEXT;
ALTER TABLE vactst ALTER COLUMN t SET STORAGE EXTERNAL;
+SELECT pg_stat_reset();
VACUUM (PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
VACUUM (PROCESS_TOAST FALSE, FULL) vactst;
-- SKIP_DATABASE_STATS option
@@ -244,6 +254,21 @@ VACUUM (SKIP_DATABASE_STATS) vactst;
VACUUM (ONLY_DATABASE_STATS);
VACUUM (ONLY_DATABASE_STATS) vactst; -- error
+-- PROCESS_MAIN option
+SELECT pg_stat_reset();
+VACUUM (PROCESS_MAIN FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+VACUUM (PROCESS_MAIN FALSE, PROCESS_TOAST FALSE) vactst;
+SELECT * FROM vactst_vacuum_counts;
+SELECT relfilenode AS main_filenode FROM pg_class WHERE relname = 'vactst' \gset
+SELECT t.relfilenode AS toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst' \gset
+VACUUM (PROCESS_MAIN FALSE, FULL) vactst;
+SELECT relfilenode = :main_filenode FROM pg_class WHERE relname = 'vactst';
+SELECT t.relfilenode = :toast_filenode FROM pg_class c, pg_class t
+ WHERE c.reltoastrelid = t.oid AND c.relname = 'vactst';
+
+DROP VIEW vactst_vacuum_counts;
DROP TABLE vaccluster;
DROP TABLE vactst;
DROP TABLE vacparted;
--
2.25.1
On Wed, Mar 01, 2023 at 10:53:59PM -0800, Nathan Bossart wrote:
I don't feel a strong need for that, especially now that we aren't
modifying params anymore.
That was mostly OK for me, so applied after tweaking a couple of
places in the tests (extra explanations, for one), the comments and
the code.
The part that improved the tests of PROCESS_TOAST was useful on its
own, so I have done that separately as 46d490a. Another thing that I
have found incorrect is the need for two pg_stat_reset() calls that
would reflect on the whole database, in the middle of a test running
in parallel of other things. As far as I understand, you have added
that to provide fresh data after a single command while relying on
vactst, but it is possible to get the same amount of coverage by
relying on cumulated counts, and that gets even more solid with all
these commands run on an independent table.
--
Michael
On Mon, Mar 06, 2023 at 04:51:46PM +0900, Michael Paquier wrote:
That was mostly OK for me, so applied after tweaking a couple of
places in the tests (extra explanations, for one), the comments and
the code.
Thanks!
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
On Mon, Mar 06, 2023 at 09:37:23AM -0800, Nathan Bossart wrote:
On Mon, Mar 06, 2023 at 04:51:46PM +0900, Michael Paquier wrote:
That was mostly OK for me, so applied after tweaking a couple of
places in the tests (extra explanations, for one), the comments and
the code.
I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is
called, 4211fbd84 changes the else into an else if [1]https://github.com/postgres/postgres/blob/master/src/backend/commands/vacuum.c#L2078. I understand
after reading the commit and re-reading the code why that is now, but I
was initially confused. I was thinking it might be nice to have a
comment mentioning why there is no else case here (i.e. that the main
table relation will be vacuumed on the else if branch).
- Melanie
[1]: https://github.com/postgres/postgres/blob/master/src/backend/commands/vacuum.c#L2078
On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote:
I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is
called, 4211fbd84 changes the else into an else if [1]. I understand
after reading the commit and re-reading the code why that is now, but I
was initially confused. I was thinking it might be nice to have a
comment mentioning why there is no else case here (i.e. that the main
table relation will be vacuumed on the else if branch).
This was a hack to avoid another level of indentation for that whole block
of code, but based on your comment, it might be better to just surround
this entire section with an "if (params->options & VACOPT_PROCESS_MAIN)"
check. WDYT?
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
On Mon, Mar 6, 2023 at 3:27 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote:
I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is
called, 4211fbd84 changes the else into an else if [1]. I understand
after reading the commit and re-reading the code why that is now, but I
was initially confused. I was thinking it might be nice to have a
comment mentioning why there is no else case here (i.e. that the main
table relation will be vacuumed on the else if branch).This was a hack to avoid another level of indentation for that whole block
of code, but based on your comment, it might be better to just surround
this entire section with an "if (params->options & VACOPT_PROCESS_MAIN)"
check. WDYT?
I think that would be clearer.
- Melanie
On Mon, Mar 06, 2023 at 03:48:28PM -0500, Melanie Plageman wrote:
On Mon, Mar 6, 2023 at 3:27 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote:
I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is
called, 4211fbd84 changes the else into an else if [1]. I understand
after reading the commit and re-reading the code why that is now, but I
was initially confused. I was thinking it might be nice to have a
comment mentioning why there is no else case here (i.e. that the main
table relation will be vacuumed on the else if branch).This was a hack to avoid another level of indentation for that whole block
of code, but based on your comment, it might be better to just surround
this entire section with an "if (params->options & VACOPT_PROCESS_MAIN)"
check. WDYT?I think that would be clearer.
Here's a patch. Thanks for reviewing.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Attachments:
process_main_readability_fix.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 580f966499..fb1ef28fa9 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -2060,23 +2060,25 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
/*
* Do the actual work --- either FULL or "lazy" vacuum
*/
- if ((params->options & VACOPT_FULL) &&
- (params->options & VACOPT_PROCESS_MAIN))
+ if (params->options & VACOPT_PROCESS_MAIN)
{
- ClusterParams cluster_params = {0};
+ if (params->options & VACOPT_FULL)
+ {
+ ClusterParams cluster_params = {0};
- /* close relation before vacuuming, but hold lock until commit */
- relation_close(rel, NoLock);
- rel = NULL;
+ /* close relation before vacuuming, but hold lock until commit */
+ relation_close(rel, NoLock);
+ rel = NULL;
- if ((params->options & VACOPT_VERBOSE) != 0)
- cluster_params.options |= CLUOPT_VERBOSE;
+ if ((params->options & VACOPT_VERBOSE) != 0)
+ cluster_params.options |= CLUOPT_VERBOSE;
- /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
- cluster_rel(relid, InvalidOid, &cluster_params);
+ /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
+ cluster_rel(relid, InvalidOid, &cluster_params);
+ }
+ else
+ table_relation_vacuum(rel, params, vac_strategy);
}
- else if (params->options & VACOPT_PROCESS_MAIN)
- table_relation_vacuum(rel, params, vac_strategy);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
On Mon, Mar 06, 2023 at 01:13:37PM -0800, Nathan Bossart wrote:
On Mon, Mar 06, 2023 at 03:48:28PM -0500, Melanie Plageman wrote:
On Mon, Mar 6, 2023 at 3:27 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
On Mon, Mar 06, 2023 at 02:40:09PM -0500, Melanie Plageman wrote:
I noticed in vacuum_rel() in vacuum.c where table_relation_vacuum() is
called, 4211fbd84 changes the else into an else if [1]. I understand
after reading the commit and re-reading the code why that is now, but I
was initially confused. I was thinking it might be nice to have a
comment mentioning why there is no else case here (i.e. that the main
table relation will be vacuumed on the else if branch).This was a hack to avoid another level of indentation for that whole block
of code, but based on your comment, it might be better to just surround
this entire section with an "if (params->options & VACOPT_PROCESS_MAIN)"
check. WDYT?I think that would be clearer.
Here's a patch. Thanks for reviewing.
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 580f966499..fb1ef28fa9 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -2060,23 +2060,25 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
I would move this comment inside of the outer if statement since it is
distinguishing between the two branches of the inner if statement.
Also, I would still consider putting a comment above that reminds us that
VACOPT_PROCESS_MAIN is the default and will vacuum the main relation.
/* * Do the actual work --- either FULL or "lazy" vacuum */ - if ((params->options & VACOPT_FULL) && - (params->options & VACOPT_PROCESS_MAIN)) + if (params->options & VACOPT_PROCESS_MAIN) { - ClusterParams cluster_params = {0}; + if (params->options & VACOPT_FULL) + { + ClusterParams cluster_params = {0};- /* close relation before vacuuming, but hold lock until commit */ - relation_close(rel, NoLock); - rel = NULL; + /* close relation before vacuuming, but hold lock until commit */ + relation_close(rel, NoLock); + rel = NULL;- if ((params->options & VACOPT_VERBOSE) != 0) - cluster_params.options |= CLUOPT_VERBOSE; + if ((params->options & VACOPT_VERBOSE) != 0) + cluster_params.options |= CLUOPT_VERBOSE;- /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */ - cluster_rel(relid, InvalidOid, &cluster_params); + /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */ + cluster_rel(relid, InvalidOid, &cluster_params); + } + else + table_relation_vacuum(rel, params, vac_strategy); } - else if (params->options & VACOPT_PROCESS_MAIN) - table_relation_vacuum(rel, params, vac_strategy);/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
- Melanie
On Mon, Mar 06, 2023 at 05:09:58PM -0500, Melanie Plageman wrote:
I would move this comment inside of the outer if statement since it is
distinguishing between the two branches of the inner if statement.
Oops, done.
Also, I would still consider putting a comment above that reminds us that
VACOPT_PROCESS_MAIN is the default and will vacuum the main relation.
I tried adding something along these lines.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Attachments:
process_main_readability_fix_v2.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 580f966499..90a42e089c 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -2058,25 +2058,32 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
save_nestlevel = NewGUCNestLevel();
/*
- * Do the actual work --- either FULL or "lazy" vacuum
+ * If PROCESS_MAIN is set (the default), it's time to vacuum the main
+ * relation. Otherwise, we can skip this part. If required, we'll process
+ * the TOAST table later.
*/
- if ((params->options & VACOPT_FULL) &&
- (params->options & VACOPT_PROCESS_MAIN))
+ if (params->options & VACOPT_PROCESS_MAIN)
{
- ClusterParams cluster_params = {0};
+ /*
+ * Do the actual work --- either FULL or "lazy" vacuum
+ */
+ if (params->options & VACOPT_FULL)
+ {
+ ClusterParams cluster_params = {0};
- /* close relation before vacuuming, but hold lock until commit */
- relation_close(rel, NoLock);
- rel = NULL;
+ /* close relation before vacuuming, but hold lock until commit */
+ relation_close(rel, NoLock);
+ rel = NULL;
- if ((params->options & VACOPT_VERBOSE) != 0)
- cluster_params.options |= CLUOPT_VERBOSE;
+ if ((params->options & VACOPT_VERBOSE) != 0)
+ cluster_params.options |= CLUOPT_VERBOSE;
- /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
- cluster_rel(relid, InvalidOid, &cluster_params);
+ /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
+ cluster_rel(relid, InvalidOid, &cluster_params);
+ }
+ else
+ table_relation_vacuum(rel, params, vac_strategy);
}
- else if (params->options & VACOPT_PROCESS_MAIN)
- table_relation_vacuum(rel, params, vac_strategy);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
On Mon, Mar 6, 2023 at 5:43 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
On Mon, Mar 06, 2023 at 05:09:58PM -0500, Melanie Plageman wrote:
I would move this comment inside of the outer if statement since it is
distinguishing between the two branches of the inner if statement.Oops, done.
Also, I would still consider putting a comment above that reminds us that
VACOPT_PROCESS_MAIN is the default and will vacuum the main relation.I tried adding something along these lines.
LGTM.
On Mon, Mar 06, 2023 at 06:12:36PM -0500, Melanie Plageman wrote:
LGTM.
- * Do the actual work --- either FULL or "lazy" vacuum
+ * If PROCESS_MAIN is set (the default), it's time to vacuum the main
+ * relation. Otherwise, we can skip this part. If required, we'll process
+ * the TOAST table later.
Should we mention that this part could be used for a toast table once
we've already looped once through vacuum_rel() when toast_relid was
set? VACOPT_PROCESS_MAIN is enforced a few lines down the road,
still..
--
Michael
On Tue, Mar 07, 2023 at 09:20:12AM +0900, Michael Paquier wrote:
- * Do the actual work --- either FULL or "lazy" vacuum + * If PROCESS_MAIN is set (the default), it's time to vacuum the main + * relation. Otherwise, we can skip this part. If required, we'll process + * the TOAST table later.Should we mention that this part could be used for a toast table once
we've already looped once through vacuum_rel() when toast_relid was
set? VACOPT_PROCESS_MAIN is enforced a few lines down the road,
still..
That did cross my mind, but I was worried that trying to explain all that
here could cause confusion.
If PROCESS_MAIN is set (the default), it's time to vacuum the main
relation. Otherwise, we can skip this part. If processing the TOAST
table is required (e.g., PROCESS_TOAST is set), we'll force
PROCESS_MAIN to be set when we recurse to the TOAST table so that it
gets processed here.
How does that sound?
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
On Mon, Mar 06, 2023 at 04:59:49PM -0800, Nathan Bossart wrote:
That did cross my mind, but I was worried that trying to explain all that
here could cause confusion.If PROCESS_MAIN is set (the default), it's time to vacuum the main
relation. Otherwise, we can skip this part. If processing the TOAST
table is required (e.g., PROCESS_TOAST is set), we'll force
PROCESS_MAIN to be set when we recurse to the TOAST table so that it
gets processed here.How does that sound?
Sounds clear to me, thanks! Melanie, what do you think?
--
Michael
On Mon, Mar 6, 2023 at 10:45 PM Michael Paquier <michael@paquier.xyz> wrote:
On Mon, Mar 06, 2023 at 04:59:49PM -0800, Nathan Bossart wrote:
That did cross my mind, but I was worried that trying to explain all that
here could cause confusion.If PROCESS_MAIN is set (the default), it's time to vacuum the main
relation. Otherwise, we can skip this part. If processing the TOAST
table is required (e.g., PROCESS_TOAST is set), we'll force
PROCESS_MAIN to be set when we recurse to the TOAST table so that it
gets processed here.How does that sound?
Sounds clear to me, thanks! Melanie, what do you think?
Yes, sounds clear to me also!
- Melanie
On Tue, Mar 07, 2023 at 12:39:29PM -0500, Melanie Plageman wrote:
Yes, sounds clear to me also!
Here is an updated patch.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Attachments:
process_main_readability_fix_v3.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 580f966499..0acc42af2b 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -2058,25 +2058,34 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params, bool skip_privs)
save_nestlevel = NewGUCNestLevel();
/*
- * Do the actual work --- either FULL or "lazy" vacuum
+ * If PROCESS_MAIN is set (the default), it's time to vacuum the main
+ * relation. Otherwise, we can skip this part. If processing the TOAST
+ * table is required (e.g., PROCESS_TOAST is set), we'll force PROCESS_MAIN
+ * to be set when we recurse to the TOAST table so that it gets processed
+ * here.
*/
- if ((params->options & VACOPT_FULL) &&
- (params->options & VACOPT_PROCESS_MAIN))
+ if (params->options & VACOPT_PROCESS_MAIN)
{
- ClusterParams cluster_params = {0};
+ /*
+ * Do the actual work --- either FULL or "lazy" vacuum
+ */
+ if (params->options & VACOPT_FULL)
+ {
+ ClusterParams cluster_params = {0};
- /* close relation before vacuuming, but hold lock until commit */
- relation_close(rel, NoLock);
- rel = NULL;
+ /* close relation before vacuuming, but hold lock until commit */
+ relation_close(rel, NoLock);
+ rel = NULL;
- if ((params->options & VACOPT_VERBOSE) != 0)
- cluster_params.options |= CLUOPT_VERBOSE;
+ if ((params->options & VACOPT_VERBOSE) != 0)
+ cluster_params.options |= CLUOPT_VERBOSE;
- /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
- cluster_rel(relid, InvalidOid, &cluster_params);
+ /* VACUUM FULL is now a variant of CLUSTER; see cluster.c */
+ cluster_rel(relid, InvalidOid, &cluster_params);
+ }
+ else
+ table_relation_vacuum(rel, params, vac_strategy);
}
- else if (params->options & VACOPT_PROCESS_MAIN)
- table_relation_vacuum(rel, params, vac_strategy);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
On Tue, Mar 07, 2023 at 12:55:08PM -0800, Nathan Bossart wrote:
On Tue, Mar 07, 2023 at 12:39:29PM -0500, Melanie Plageman wrote:
Yes, sounds clear to me also!
Here is an updated patch.
Fine by me, so done. (I have cut a few words from the comment,
without changing its meaning.)
--
Michael